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Abstract 


This paper describes a usability study of proposed cockpit interfaces 
to support Airborne Precision Spacing (APS) operations for aircraft 
performing dependent parallel approaches (DPA). NASA has proposed 
an airborne system called Pair Dependent Speed (PDS) which uses their 
Airborne Spacing for Terminal Arrival Routes (ASTAR) algorithm to 
manage spacing internals. Interface elements were designed to facilitate 
the input of APS-DPA spacing parameters to ASTAR , and to convey PDS 
system information to the crew deemed necessary and/or helpful to 
conduct the operation, including: target speed, guidance mode, target 
aircraft depiction, and spacing trend indication. In the study, subject 
pilots observed recorded simulations using the proposed interface 
elements in which the ownship managed assigned spacing intervals from 
two other arriving aircraft. Simulations were recorded using the Aircraft 
Simulation for Traffic Operations Research (ASTOR) platform, a 
medium-fidelity simulator based on a modern Boeing commercial glass 
cockpit. Various combinations of the interface elements were presented 
to subject pilots, and feedback was collected via structured 
questionnaires. The results of subject pilot evaluations show that the 
proposed design elements were acceptable, and that preferable 
combinations exist within this set of elements. The results also point to 
potential improvements to be considered for implementation in future 
experiments. 



2 Abbreviations 


ADS-B 

Automatic Dependent Surveillance - Broadcast 

AFDS 

Autopilot Flight Director System 

ALT 

Altitude 

APS 

Airborne Precision Spacing 

AOC 

Airline Operations Center 

APPR 

Approach 

ASTAR 

Airborne Spacing for Terminal Arrival Routes 

ASTOR 

Aircraft Simulation for Traffic Operations Research 

ARINC429 

Aeronautical Radio, Incorporated 429 data bus 

ATC 

Air Traffic Control 

ATOL 

Air Traffic Operations Laboratory 

ATSP 

Air Traffic Service Provider 

CDA 

Continuous Descent Arrival 

CDTI 

Cockpit Display of Traffic Information 

CPDLC 

Controller Pilot Datalink Clearance 

ConOps 

Concept of Operation 

DEV 

Deviation 

DPA 

Dependent Parallel Approaches 

EFB 

Electronic Flight Bag 

EICAS 

Engine Instrument and Crew Alerting System 

EXEC 

Execute 

FAA 

Federal Aviation Administration 

FMA 

Flight Mode Annunciator 

FMC 

Flight Management Computer 

FNL 

PDS Final Mode 

HITL 

Human-in-the-Loop 

ID 

Aircraft Identification 

MCDU 

Multifunction Control and Display Unit 

MCP 

Mode Control Panel 

MFD 

Multifunction Display 

MIN 

Minimum 

MOD 

Modified 

N/A 

Not Applicable 

NASA 

National Aeronautics and Space Administration 

NAVAID 

Navigation Aid 

NCT 

No Closer Than 

ND 

Navigation Display 



NG 

NM 

PAIR 

PFD 

PDS 

PRI 

PTH 

REFAC 

RTA 

RWY 

SEC 

SPD 

STA 

TOD 

VNAV 


Next Generation 
Nautical Miles 
PDS Paired Mode 
Primary Flight Display 
Pair Dependent Speed 

Primary Spacing Aircraft, also kn own as Active Spacing Aircraft 
Path 

Reference Aircraft, also known as Target Aircraft 

Required Time of Arrival 

Runway 

Seconds 

Speed 

Scheduled Time of Arrival 
Top of Descent 
Vertical Navigation 


3 Introduction 

Airborne Precision Spacing (APS) is an operational concept where the control of the aircraft’s 
speed is delegated by the air traffic service provider (ATSP) to the flight crew in order to 
precisely achieve an assigned inter-aircraft spacing. The concept allows the flight crew to make 
minor speed adjustments based on cues from an on-board system while flying a continous 
descent arrival (CDA). The concept is intended to address both capacity and efficiency issues 
facing the air transportation system by enabling the merging of efficient trajectories by a variety 
of aircraft into a precisely-spaced arrival stream in the terminal area. 

Airborne systems that could be used for generating the speed commands have been under 
development for more than 10 years[ 1 ] [2] [3] . Fast-time simulations and human-in-the-loop 
(HITL) studies have been conducted during that period which have verified the effectiveness of 
the algorithms in achieving precise inter-aircraft spacing in the tenninal environment[4][5]. The 
current NASA instantiation of this algorithm is called Airborne Spacing for Terminal Arrival 
Routes (ASTAR), and the airborne system is called Pair Dependent Speed (PDS). 

As the development of airborne spacing systems progresses, so also does research into the 
procedures and human interfaces required to conduct spacing operations. EuroControl has 
already proposed flight deck user requirements for airborne spacing[6]. The HITL studies 
already mentioned also sought to detennine if the procedures and cockpit interfaces proposed for 
conducting APS operations were acceptable. A recent human-in-the-loop (HITL) experiment 
conducted at NASA Langley’s Air Traffic Operations Laboratory (ATOL) applied a prototype 
PDS system and associated procedures to the conduct of paired CDA profiles to a single runway. 
The results of this study showed that the procedures and interfaces used for airborne spacing can 
achieve the desired results with little negative performance impact over conventional CDAs[5], 



NASA has been working to extend the APS concept to support dependent parallel approaches 
(DP A), where the runways are close enough together that spacing must be managed between 
aircraft arriving on both runways[7]. This requires a redesign of the way the crew will interface 
with the aircraft, including the traffic display, speed guidance display, data entry and crew 
notifications. The focus of the study described in this report was to design, test, and refine crew 
interface concepts for APS-DPA operations. Two crew interface systems were investigated: a 
system with the interfaces fully integrated with the standard cockpit displays, and a system with 
the application and interfaces hosted on an Electronic Flight Bag (EFB) 1 . 

For this study, two interface concepts were designed, one suitable for a typical modem glass 
cockpit and the other for an EFB, and included means for inputting spacing control parameters, 
the display of PDS-commanded speed and PDS mode, the display of target aircraft and spacing 
trend indication. NASA’s Aircraft Simulation for Traffic Operations Research (ASTOR) was 
utilized as a platfonn on which to implement and display these concepts for the study. The study 
sought to maximize the insight gained from the pilot evaluations by developing alternative 
versions of the interfaces such that pilots could make comparisons and express preferences for 
either individual design elements or combinations of the elements. 

This report describes the conceptual design of the proposed APS interfaces for the fully 
integrated system, the methodology used for the conduct of the experiment, and the results of the 
pilot evaluations. The EFB interface concepts are also described herein, but were not intended to 
be part of the usability study (see Appendix C: Electronic Flight Bag Interface Concepts for 
APS-DPA). Final conclusions and recommendations are made based on analysis of the pilot 
evaluation data. 

4 Fully Integrated APS Interface Concepts 

The goal for the fully integrated version of the APS-DPA interface was to develop interfaces to 
allow pilots to perfonn the following functions: 

1 . manage voice or datalink reception of spacing clearances; 

2. initialize PDS with spacing clearance parameters; 

3. verify validity of spacing clearance parameters before executing; 

4. engage PDS target speed to the autopilot while also flying CDA profde; 


1 An EFB (FAA's Advisory Circular - AC120-76A[10]), is an electronic display system in addition to the standard 
cockpit displays that displays a variety of aviation data, automates basic calculations (e.g., performance data, fuel 
calculations, etc.), and/or presents various graphical information to help flight crews perform flight management 
tasks more easily and efficiently. 



5. monitor progress of APS-DPA, including: 

a. PDS target speed; 

b. PDS operation mode; 

c. Target aircraft position and information; 

d. Spacing deviation and trend information; and 

e. Off-nominal indications. 

6. disengage PDS from autopilot and/or cancel spacing clearance. 

Concepts were developed which would modify the operation of several typical glass cockpit 
displays, interfaces, and their underlying control software. These concepts were implemented in 
a special ASTOR software build created for the study to support simulations, and included 
modifications to the following: 

• Engine Instrument & Crew Alerting System/Multifunction Display (EICAS/MFD); 

• Mode Control Panel (MCP); 

• Multifunction Control and Display Unit (MCDU); 

• Primary Flight Display (PFD); and 

• Navigation Display (ND). 

The focus of the study was to obtain pilot evaluations for the design elements developed for the 
PFD and the ND. The modifications made to the EICAS/MFD, MCP and MCDU were necessary 
to integrate the APS-DPA functionality and enhance the realism of the spacing scenarios 
presented on the ASTOR, but were not the focus of pilot evaluations. Implementation details of 
the EICAS/MFD, MCP and MCDU concepts are discussed in Appendix B: Supplemental Design 
Concepts for Fully Integrated Implementation. 

4.1 APS Interface Concepts for Primary Flight Display 

The PFD typically displays tactical information to the crew. Alternative design concepts were 
proposed to use the PFD to present the PDS-commanded target speed and annunciation of the 
active PDS guidance mode. 

4.1.1 PDS Target Speed Indication 

During spacing operations, ASTAR calculates a target speed (also referred to as PDS target 
speed) to be met by the aircraft in order to achieve the spacing interval. Two different concepts 
for displaying the PDS target speed were implemented for evaluation in this study, shown in 
Figure 4-1. 




Figure 4-1: PDS Target Speed Concepts: Text (Left); Speed Bug (Right). 


In the first concept, the PDS target speed appeared as a textual indication in a dedicated area of 
the upper right corner of the PFD. The PDS target speed text appeared in white until PDS was 
engaged to the autopilot, when the text changed to green. Changes to PDS target speed were 
highlighted for 10 seconds by a flashing box surrounding the text. This concept was identical to 
the implementation of PDS target speed in the preceding APS HITL study. 

This concept was considered acceptable by subject pilots participating in the previous 
experiment who were tasked with manually entering the PDS target speed into the MCP speed 
window. However, the concept had not yet been applied to the conduct of APS-DPA operations, 
nor had it been used in a fully integrated, autopilot-flown profile. 

In the second concept, the PDS target speed was indicated by a filled green “speed bug” on the 
speed tape of the PFD. The green bug appeared when PDS was providing valid speed guidance, 
regardless of the state of autopilot engagement. When the PDS target speed was engaged to the 
autopilot, the implementation caused the green bug and the normal commanded-speed bug 
(magenta) to coincide. The green bug was displayed all the way to touchdown unless PDS was 
terminated. 

It was thought that a graphical PDS target speed indicator provided a more visual, contrasting 
alternative to the textual display. By juxtaposing the PDS target speed bug and the nonnal 
autopilot commanded speed bug such that neither could be obscured, the pilot could quickly 
determine whether the two coincided or diverged. In autothrottle mode, this allowed the pilot to 
anticipate speed changes from PDS; if manual speed intervention were used (which was not part 
of this study) the pilot could adjust the commanded speed until the indicators coincided. 
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4.1.2 PDS Mode Annunciation 


During an APS operation, PDS speed guidance transitions through several different modes of 
operation. Beyond ADS-B range of the target aircraft, the Required Time of Arrival (RTA, 
formerly referred to in the APS concept as STA for Scheduled Time of Arrival) mode drives the 
target speed. The Paired mode is active once all of the data is available to begin spacing relative 
to the target aircraft. The Final mode indicates PDS speed guidance has transitioned to a 
stabilized final approach speed. Two other modes were defined in the ConOps; PDS Profile 
mode (speed guidance without interval management) and PDS Speed Reversion mode (off- 
nominal conditions suspending ASTAR guidance). Neither were implemented for this study, but 
concepts were proposed for their display. Two different concepts were implemented for the 
display of PDS mode annunciation, shown in Figure 4-2. 



Figure 4-2: PDS Mode Annunciation Concepts: Text (Left); FMA (Right). 


In the first concept, the PDS mode appeared as a textual indication in a dedicated area of the 
upper right comer of the PFD. The PDS mode text appeared in white until PDS was engaged to 
the autopilot, when the text changed to green. Changes to PDS mode were highlighted for 10 
seconds by a flashing (white or green) box surrounding the text. This concept was identical to 
the implementation of PDS target speed in the preceding APS HITL study. 

This concept was considered acceptable by subject pilots participating in the previous 
experiment, however, it had not yet been applied to the conduct of APS-DPA operations, nor had 
it been used in a fully integrated, autopilot-flown profile. 

In the second concept, PDS mode annunciation was integrated into the Flight Mode Annunciator 
(FMA) of the PFD. The FMA is used to annunciate autoflight settings and modes of the 
autopilot. To indicate PDS autoflight modes, the pitch mode segment of the FMA was divided 
into four (4) quadrants, as shown in Figure 4-3. The leftmost upper and lower quadrants of the 
pitch mode segment displayed the vertical autopilot mode currently selected and armed, as 






normal. The modified right upper and lower quadrants of the pitch mode segment displayed the 
active and armed PDS modes, respectively. 



1 r 



Figure 4-3: FMA Pitch Segment Modified to Display PDS Modes 


It was thought that annunciating PDS mode with other autopilot modes in the FMA was more 
consistent with an integrated approach than the ad hoc presentation of the first concept. (The 
integration of PDS guidance to the autopilot is also discussed in Appendix B: Supplemental 
Design Concepts for Fully Integrated Implementation.) Table 4-1 shows several examples of the 
concept implementation during typical phases of an APS-DPA operation. The following 
conventions established for FMA annunciation of existing modes permitted enhancements to the 
PDS mode display not implemented in the first concept. For example, display of the next PDS 
mode in sequence was implemented on the line reserved for armed modes on the FMA. Further, 
a convention exists to display degraded autopilot modes on the FMA, and it could be applied to 
off-nominal PDS conditions, such as PDS Speed Reversion mode. (An example of this concept is 
also shown, but was not implemented in ASTOR.) 


Table 4-1: PDS Mode Annunciation Examples; FMA. 


PDS 

FMA 

Example 

Description 

PDS RTA 

When PDS mode 

Mode 

changes, green box 

Engaged 

appears for 10 
seconds; Pair mode 
armed 

PDS Pair 

PDS Final mode 

Mode 

Engaged 

armed; G/S armed 

PDS Final 

VNAV disengaged; 

Mode 

Engaged 

G/S engaged 

PDS 

PDS off-nominal 

Mode 

condition (Speed 

Degraded 

Reversion mode) 


FMA Pitch Mode 
Depiction 



G/S FNL 









4.2 APS Interface Concepts for the Navigation Display 

The ND presents strategic information to the crew, so this design concept used the ND to present 
the “big picture” of the APS-DPA operation. Concepts were developed for depicting one or two 
target aircraft involved in the spacing operation. Concepts were also developed for depicting the 
spacing deviation output from PDS in the fonn of a trend indicator, which might be used as a 
strategic tool for monitoring the operation’s progress. 

4.2.1 Target Aircraft Depiction 

The concepts for displaying target aircraft relied on existing or proposed standards for Cockpit 
Display of Traffic Information (CDTI) symbols. Both DO-317 (Minimum Operational 
Performance Standards for Aircraft Surveillance Applications System) [8] and a draft symbol set 
appendix for DO-289 (Aircraft Surveillance Applications Minimum Aviations System 
Performance Standards) [9] were used as references. The standards impart specific meanings to 
CDTI symbols and colors, which were applied to the paradigm of an on-board application 
supporting the APS-DPA operation. 

Two similar concepts were proposed, shown in Figure 4-4. For both concepts, the standard CDTI 
symbol for depicting ADS-B targets was a chevron, and in particular, a green chevron with a 
shape-hugging border indicated that the target was coupled to an on-board application (in this 
case, PDS). For both concepts, ND conventions for displaying CDTI symbols were implemented, 
such that the aircraft ID of the target aircraft was displayed, subject to pilot control. Further, if 
the ND range selector put the CDTI symbol(s) out of range, convention dictated that a half- 
symbol for the symbol appeared on the outer range ring of the ND at the appropriate bearing. 
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Figure 4-4: Target A/C Concepts: Coupled Only (Left); Coupled with Data Block (Right). 





The first concept displayed the target aircraft as the green “double” chevrons. No distinction was 
made between the two target aircraft other than their associated IDs, i.e., which aircraft was the 
precision or “no closer than” (NCT) target, or which aircraft was the active spacing target. This 
was a basic concept for displaying the target aircraft with a minimum of clutter on the ND. It was 
thought that pilots wishing to distinguish between precision and NCT targets could derive this by 
correlating the aircraft ID and the original spacing clearance parameters on the PDS main menu 
page of the MCDU. Similarly, the active spacing aircraft ID was depicted (in magenta) on the 
same MCDU page, see the implementation detail in Appendix B: Supplemental Design Concepts 
for Fully Integrated Implementation”. 

The second concept interpreted the standards further to make a distinction between the two 
spacing targets by adding a green “selected” circle around the active spacing target. This 
enabled another enhancement in the fonn of a data block associated with the selected target. The 
data block always appeared in the lower left portion of the ND and contained infonnation 
associated with the selected target aircraft, including the aircraft ID, aircraft type, spacing 
interval type, spacing deviation and landing runway. This concept was thought to provide 
significantly more information about the target aircraft, at the expense of additional ND clutter. 
The justification for displaying the additional information was to conveniently present strategic 
information on the progress of the spacing operation that would otherwise only be available 
through the PDS menu pages in the MCDU. (The MCDU has several other uses, and use of the 
data block could avoid interrupting them to look for spacing information.) 

Note that while the first concept could be considered to be a selectable subset of the second, the 
ability to select ND views was not available in the study’s scenario presentation, so the concepts 
were presented separately. 

4.2.2 Trend Indication 

Trend indication is potentially useful strategic infonnation during phases of APS when PDS is 
managing the spacing interval. The trend indicator concept proposed synthesizing PDS outputs 
to depict the aircraft’s position relative to spacing limits. A spacing operation can be conducted 
within recoverable limits of spacing deviation from the specified spacing interval. Beyond these 
limits, the deviation is tenned excessive and the spacing interval can no longer be achieved 
without also exceeding programmed PDS limits. The spacing deviation (in seconds) from 
nominal was displayed on the MCDU’s PDS main menu page (and also on certain target aircraft 
depictions) but the maximum limits of deviation were not displayed. Two different concepts 
were developed for integrating both pieces of infonnation into a trend indicator, shown in Figure 
4 - 5 . 





Figure 4-5: Trend Indicator Concepts: Conformance Box (Left); Deviation Scale (Right). 

In the first concept, the excessive spacing deviation limits were used to draw pictorial 
“conformance box” brackets around the ownship symbol on the ND. The box was positioned 
around the nose of the ownship symbol relative to the spacing deviation from nominal. Positive 
spacing deviations shifted the box so that the forward brackets moved away from the nose of the 
ownship; negative spacing deviations shifted the forward brackets towards the nose of the 
ownship. There was no center position depicted; as long as the nose of ownship remained within 
the conformance box, the spacing deviation was within recoverable limits. 

The conformance box boundaries were range-based (time-based deviation limits from PDS 
converted to distance), thus the size of the depicted conformance box varied in accordance with 
the ND range selector. It was thought that this concept offered an intuitive, scan-friendly option 
for viewing the trend information because it was aligned with the ownship and the course line. 

The second concept proposed a deviation indicator presentation in the lower left corner of the 
ND. The deviation outputs from PDS were synthesized to display the aircraft’s spacing deviation 
relative to the excessive spacing limits with a green pointer on a fixed scale. A positive spacing 
deviation moved the pointer up the scale, and a negative spacing deviation moved the pointer 
down. (A modification to this concept is discussed in Section 6.5.) Unlike the first concept, the 
size of the scale remained constant, therefore the sensitivity increased as the PDS limits 
decreased and vice-versa. Like the first concept, there was no zero deviation center indicator. It 
was thought that this design concept would seem familiar to current commercial pilots, and 
therefore be met with some acceptance, because a similar Vertical Navigation (VNAV) deviation 
scale is depicted in the lower right comer of the ND to indicate altitude deviations relative to the 
profile being flown. 
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5 Methodology 

5.1 Experiment Objectives 

The primary objective of the experiment was to evaluate the usability and acceptability of the 
interfaces designed to support APS-DPA operations. It was hypothesized that pilots would find 
all versions of the interfaces presented to be both usable and acceptable, and that insight would 
be gained from analyzing the questionnaire responses by subject pilots in the following areas of 
interest: 

• importance of the proposed interface to the conduct of the operation; 

• usability of proposed interfaces; 

• preferences in the proposed display interfaces; 

• preferences in combinations of the proposed interfaces; 

• design anomalies; and 

• suggestions for improvement. 

Since this was a study in which no interaction with the recorded scenario was possible, there was 
no attempt to evaluate subjective workload, procedures, or performance. 

5.2 Dependent Measures 

The dependent variables for the study consisted of subject pilot evaluations of the interface 
design elements, collected via structured questionnaires administered after individual test runs 
and also after the complete session. The questionnaires were designed to gather the following 
types of infonnation. 

• Usability: Were the design elements understandable? Were they intuitive, user-friendly? 

• Usefulness: Were the design elements needed or not? After seeing them, if they were 
subsequently removed, would the subject pilot miss any of them? Did they impact the 
situation awareness of the APS-DPA operation? 

• Acceptability: Did the design elements make the subject pilot feel more comfortable about 
the APS-DPA operation? Was the subject pilot confident about the status of the operation? 

• Improvements: Assuming the proposed concepts were useful, what could be done to enhance 
the effectiveness, appearance or other aspects of the design elements? 

5.3 Experiment Scenario Design 

The concept for the experiment was to present the proposed interface concepts in a manner best 
suited to obtaining subject pilot evaluations on their acceptability and usability. The ASTOR 
platform was chosen for its ability to realistically represent the proposed APS interfaces as they 
would appear on modern Boeing commercial glass cockpit displays. Custom ASTOR software 
was developed to implement the APS interface concepts, and modified spacing scenarios from 
the previous ATOL APS experiment were used to run simulations with the ASTOR software and 
generate recordings of a simulated spacing operation from the cockpit perspective of the 
simulated aircraft. 



Although this approach required more effort than traditional storyboarding techniques for 
presenting new display concepts, it was deemed to have several advantages. First, the recorded 
spacing scenario offered a continuous, full-motion representation of the interface concepts as 
they appeared during the spacing operation, enhancing the realism and affording the subject 
pilots an opportunity to evaluate the interfaces appearance at any point in the scenario. Second, 
the ASTOR programming modifications were reusable for future ATOL demonstrations and/or 
experiments at the completion of this study. A key feature of the ASTOR modifications was the 
ability to control the APS interface design elements via a configuration file, such that any 
combination of the design elements could be selectively enabled for a particular simulation. This 
feature allows these elements to be quickly modified to evaluate their usability in future spacing 
scenarios. 

The traffic scenario for the recorded simulation utilized models which match CDA operations at 
the Louisville Standiford Airport (SDF). An existing traffic scenario which merged two east- 
bound streams into a single CDA to Runway 17R was modified to separate the streams and 
simulate dependent parallel approach operations to Runway 17R and Runway 17L. A list of 
simulated test conditions is included below. Simulated winds were set to zero (no wind) to 
enhance the repeatability of the scenarios from run to run. 

• Winds: No wind, all altitudes 

• Aircraft type: Heavy, two-engine, narrow-body transport aircraft 

• ASTOR Build: OpTech_ASTAR10_2010SEP16 

• ASTAR version: 10 

The scenario consisted of six aircraft (ID = NASA xxx), each assigned to fly one of the two 
published CDAs: 1) the SEA BISCUIT ONE (CBSKT1) ARRIVAL to Runway 17R, and 2) the 
CALKS ONE (CALKS 1) ARRIVAL to Runway 17L. The lead aircraft in the CALKS ONE 
arrival stream was NASA 907, which autonomously flew the CDA to Runway 17L with a 
Required Time of Arrival (RTA) but without an assigned spacing interval. The lead aircraft in 
the SEA BISCUIT ONE arrival stream was NASA 893, which autonomously flew the CDA to 
Runway 17R with a RTA and without an assigned spacing interval. The remaining aircraft in the 
scenario autonomously flew the CDAs and spacing interval assignments specified in Table 5-1. 
See Figure 5-1 for a pictorial diagram of the spacing scenario which shows the spacing intervals 
for the ASTOR station (NASA 995) used as the ownship in the experiment presentation. 


Table 5-1: Spacing Scenario Clearance Parameters. 


Aircraft ID 

CDA 

Runway 

RTA 

Precision 

Interval 

Target 

Aircraft 

NCT 

Interval 

Target 

Aircraft 

NASA 907 

CALKS 1 

17L 

06:24:13Z 





NASA 859 

CALKS 1 

17L 

06:26:52Z 

180 sec 

NASA 907 



NASA 91 7 

CALKS 1 

17L 

06:32:53Z 

140 sec 

NASA 893 



NASA 893 

CBSKT1 

17R 

06:25:32Z 





NASA 995 

CBSKT1 

17R 

06:28:16Z 

165 sec 

NASA 893 

90 sec 

NASA 859 





The intrinsic recording facilities of ASTOR were utilized to capture data necessary for playback 
of the multi-aircraft spacing scenarios for the usability study presentations. This facility enabled 
a portable demonstration displaying multiple aircraft while using only one ASTOR station. The 
use of a recorded simulation file ensured that the simulated scenario would appear identically for 
each presentation. The simulations were recorded in a laboratory similar to the ATOL, equipped 
with several ASTORs and a scenario traffic generator. The experimental software build used for 
the simulations allowed the ASTOR to: 1) simulate reception of a spacing clearance via datalink; 
2) parse the clearance and load the spacing parameters to the MCDU PDS menu pages 
(Appendix B: Supplemental Design Concepts for Fully Integrated Implementation); and 3) 
execute the spacing instructions by autonomously flying the PDS target speed. After the scenario 
was run to completion, the recorded file from the ASTOR representing NASA 995 was saved for 
playback during the experiment sessions. 

The procedure for recording the experimental scenario was to start the simulation with all 
ASTOR stations (aircraft) at a point prior to top of descent (TOD). Aircraft then received spacing 
clearances via datalink, also before TOD. The test operator visited each ASTOR station to 
manually load the received clearance parameters to ASTAR via the MCDU PDS menu pages. 

The verified clearance was then executed and the autopilot was manually engaged by selecting 
the “PDS” engage function on the MCP. The manual nature of serially starting all six of the 


ASTOR stations led to minor timing variations in the eight (8) recorded scenarios, but these 
variations were largely damped out as the scenario progressed. 

In order to program the ASTOR representing NASA 995 with the NOT spacing interval 
necessary to simulate APS-DPA, the test operator manually entered the NCT interval parameters 
via the MCDU PDS menu interface. Again, this was done prior to the TOD for NASA 995. Once 
the initial clearance was modified with the NCT interval, the test operator manually executed the 
clearance via the MCDU. 

5.4 Experimental Protocol 

An experiment session consisted of several presentations displayed on a workstation consisting 
of a computer monitor and a mouse. The associated computer was programmed with the ASTOR 
software, which simulated the operation of a modem commercial Boeing glass cockpit. Before 
the ASTOR software was started, the subject pilot was given a pre-experiment briefing on APS- 
DPA concepts and the proposed design concepts that were to be evaluated. 

The bulk of the session consisted of nine runs, each of which presented the subject pilot with a 
recorded spacing scenario replayed on the ASTOR platform. The experimental runs each 
simulated an APS-DPA spacing operation in which the ownship (NASA 995) was assigned both 
a precision interval and an NCT interval. Played back at normal speed, the recorded simulations 
would have lasted approximately twenty-four (24) minutes from TOD to touchdown. However, 
the ASTOR playback facility allowed the speed of playback to be varied by the test operator. 
This facility was used to fast-forward the playback through periods of relative inactivity, and 
each run was shortened to approximately five to seven minutes. 

During the first familiarization run (RO), the subject pilot was asked to interact with the ASTOR 
by using the mouse to enter a typical spacing clearance into the MCDU pages specifically 
designed for this exercise. In the remaining runs (R1-R8), the subject pilots were asked to only 
observe the presentation as the spacing scenario progressed. 

A post-run questionnaire was administered to capture the subject pilot’s evaluation of the design 
elements presented. After all of the test runs were completed, a final questionnaire was 
administered in which overall impressions of the test were captured. 

In an ideal experiment, only one independent variable would change per ran in order to best 
associate its interrelation to the dependent variables. This approach was considered both 
impractical and unnecessary for this usability study, as it would have taxed the desired 2-3 hour 
time limit for the experiment, and it was believed the grouping of certain design elements would 
not significantly compromise the subjective data obtained. See Table 5-2 for a description of test 
runs, detailing the design elements which were grouped for each run. The runs were given a run 
number so they could be correlated to the post-run questionnaire data. Although the test runs 
were numbered, the run order was randomized for each test session to prevent unintended bias 
introduced by a set order. 



Table 5-2: Test Run Matrix Showing Display Options Enabled 







R1 

Text 

Text 

Coupled Only 

None 

R2 

Text 

FMA 

Coupled Only 

None 

R3 

Speed Bug 

Text 

Coupled Only 

None 

R4 

Speed Bug 

FMA 

Coupled Only 

None 

R5 

Text 

Text 

Coupled Only 

Conformance Box 

R6 

Text 

Text 

Coupled Only 

Deviation Scale 

R7 

Text 

Text 

Coupled w/Data 

Conformance Box 

R8 

Text 

Text 

Coupled w/Data 

Deviation Scale 


Run RO: This run was part of every experiment in order to introduce the subject pilot to the 
MCDU PDS menu pages added to monitor and control the spacing operation, and also 
familiarize the subject pilot with the specific spacing clearance(s) which were part of the 
recorded scenario. The subject pilot was given specific tasks to accomplish in these menu pages, 
which included: 

• Identifying the spacing aircraft and intervals; 

• Identifying the landing runway(s) for the spacing aircraft; 

• Detennining whether the clearance can be accepted (i.e., ASTAR algorithm is providing 
speed guidance); 

• Modifying a spacing parameter as requested by the investigator; 

• Executing the spacing clearance. 

Runs R1-R4: These runs were made holding design elements for target aircraft and trend 
indication constant, while varying the combinations for displaying PDS target speed and PDS 
mode, as shown in Table 5-2. 

Runs R5-R8: These runs were made holding design elements for PDS target speed and PDS 
mode constant, while varying the combinations for displaying target aircraft and trend indication, 
as shown in Table 5-2. 

5.5 Subject Pilots 

The subject pilots for this study were either current or former commercial airline pilots with 
experience in one or more of the following aircraft: 1) Boeing 777; 2) Boeing 737 NG (Next 
Gen); 3) Boeing 767-400; and 4) Boeing 747-400. The ASTOR platfonn’s representation of a 
modern Boeing glass cockpit was sufficiently familiar to all subject pilots to require no 
additional training other than a briefing on the new APS interfaces, and the experimental 
scenarios and procedures. 

Volunteers were solicited by contacting the Airline Operations Centers (AOCs) for United 
Airlines at the Chicago (ORD) and Dulles (IAD) airports. These pilots were tested in-domicile. 




Additional local pilots were solicited for dry-run experiments held at or near the Contractor’s 
office in Hampton, VA. These pilots had the same experience as listed above, and flew (or had 
recently flown) for Airtran Airlines, American Airlines, and Southwest Airlines. All subject 
pilots received a small stipend for participating. 

6 Results and Discussion 

The following is a discussion of the experiment objectives and dependent measures obtained 
from analysis of the subject pilot responses to the test questionnaires. The full text of the 
questionnaires, as well as a complete set of results for the questionnaires, is shown in Appendix 
A: Tabulated Experimental Results. 

6,1 Subject Pilot Ratings for Usability of Proposed Interfaces 

The post-run questionnaires asked subject pilots to rate their ability to identify specific 
information presented in the spacing scenario, using a scale of 6 “very easy to identify” to 1 “not 
able to identify at all”. If it was believed the information was missing from a particular run, the 
subject pilot was asked to answer “Not Applicable” (“N/A”), which was interpreted as missing 
data. These data were correlated with the elements presented in that run (see Figure 6-1) to 
determine the rating for a particular design element option (for example, text vs. green bug 
option for PDS target speed.) Since the test run matrix was not balanced (some design options 
were shown more frequently than others), the sample size (N) for each element option is noted in 
the following results analysis. 

The post-run questionnaire also asked subject pilots to rate the helpfulness of each design 
element in understanding the spacing operation, using a scale of 5 “very helpful” to 1 “not 
helpful”. These data were correlated with the elements presented in that run as above. 

For PDS target speed, the text option was presented to subject pilots in runs Rl, R2, R5, R6, R7 
and R8; the speed bug option was presented in runs R3 and R4. After each run, subject pilots 
were asked to rate their ability to identify 1) current PDS target speed and 2) changes to PDS 
target speed. The pertinent data collected from this question is summarized in Table 10-2 and 
Table 10-3. As shown in Figure 6-1, the average ratings for the text option were 5.40 (SD= 0.65, 
N=150) and 5.03 (SD=0.87, N=150), respectively. The average rating for the green speed bug 
were 5.22 (SD=0.92, N=50) and 5.00 (SD=1.06, N=50), respectively. In response to the 
helpfulness question (see Table 10-20), the average rating was 4.22 (SD=0.66, N=147 2 ) for the 
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The missing response(s) were due to an internet connection drop while the questionnaire was being administered. 



2 

text option, and 4.08 (SD=0.99, N=49") for the green speed bug. This indicates subject pilots had 
a high degree of confidence in the usability of either design option, with the text option rating 
slightly higher for all categories. 


Helpfulness 
(Scale 1-5) 


Changes to Target Speed 
(Scale 1-6) 


Current Target Speed 
(Scale 1-6) 



Average Rating 


■ speed bug 

■ text 


Figure 6-1: PDS Target Speed Option Average Ratings. 


For PDS mode, the text option was presented to subject pilots in runs Rl, R3, R5, R6, R7 and 
R8; the FMA option was presented in runs R2 and R4. After each run, subject pilots were asked 
to rate their ability to identify 1) current PDS mode and 2) changes to PDS mode. The pertinent 
data collected from this question is summarized in Table 10-4 and Table 10-5. As shown in 
Figure 6-2, the average ratings for the text option were 5.17 (SD= 0.77, N=150) and 4.83 
(SD=0.88, N=149 2 ), respectively. The average ratings for the FMA option were 5.16 (SD=1.07, 
N=50) and 4.96 (SD=1.08, N=50), respectively. Subject pilots were also asked to rate their 
ability to identify the next PDS mode in sequence. In this case, only the FMA option provided 
this information, so subject pilots should have responded with “N/A” for the runs with the text 
option. A significant number of subject pilots did not respond with “N/A” (many responded 
instead with “not able to identify”), indicating some confusion regarding the instructions. The 
pertinent data collected from this question is summarized in Table 10-6. The average rating to 
this question was 3.23 (SD=2.03, N=120) for the text option, and 4.85 (SD=1.43, N=46) for the 
FMA option. If the N/A responses are converted to “not able to identify”, the average rating 
becomes 2.78 (SD=2.02, N=150) for the text option. Both of these results are significantly lower 
than for the FMA option. These data indicate subject pilots had a high degree of confidence in 
their ability to identify PDS mode and mode changes for either design option, and a higher 
degree of confidence in their ability to identify the next mode in sequence for the FMA option. In 
response to the helpfulness question (see Table 10-21), the design alternatives were rated equally 
high, with average ratings of 4.00 for each alternative. 


Helpfulness 
(Scale 1-5) 

Next Mode in Sequence 
(Scale 1-6) 

Changes to Mode 
(Scale 1-6) 

Current Mode 
(Scale 1-6) 



Average Rating 


■ FMA 

■ text 


Figure 6-2: PDS Mode Annunciation Option Average Ratings. 


For target aircraft depiction, the “coupled only” option was presented to subject pilots in runs 
Rl, R2, R3, R4, R5, and R6; the “coupled with data block” option was presented in runs R7 and 
R8. Subject pilots were asked to rate their ability to identify several characteristics of the target 
aircraft. Some of these characteristics were only identifiable in the data block, so subject pilots 
should have responded with “N/A” for the runs where the information was not present. The lack 
of “N/A” responses led to the hypothesis that subject pilots may have identified some of this 
information from the MCDU, which was “parked” on the PDS main menu page during the 
recorded scenarios. (In support of this hypothesis, several subject pilots responded correctly for 
target aircraft characteristics available ONLY from the data block and not the MCDU page. For 
instance, assigned runway and aircraft type together received fifty-six of the fifty-seven “N/A” 
responses in the “coupled only” runs, indicating that most subject pilots correctly responded that 
the information was missing.) This menu page would have allowed the subject pilot to determine 
the active spacing aircraft, spacing interval type and spacing deviation without the need for the 
data block. This may have skewed the intended results of the questions about these 
characteristics. The pertinent data collected from this question is summarized in Table 10-7 
through Table 10-18, and Table 10-22. The average ratings for rating the usability of target 
aircraft characteristics and the overall helpfulness of the design alternatives are shown pictorially 
in Figure 6-3. The average ratings consistently favor the “coupled with data block” option over 
the “coupled only” option, indicating that subject pilot’s found characteristic(s) of the fonner 
more useful and helpful. The perceived importance of these characteristics is discussed in the 
next section. 
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How Many 



Average Rating 


■ coupled w/data block 

■ coupled only 


Figure 6-3: Target Aircraft Depiction Option Average Ratings. 


For the trend indicator, the “conformance box” option was presented to subject pilots in runs R5 
and R7, the “deviation scale” option was presented in runs R6 and R8, and there was no trend 
indicator displayed on runs Rl, R2, R3, and R4. After each run, subject pilots were asked to rate 
their ability to determine and understand the aircraft’s progress towards achieving the assigned 
spacing goal. The pertinent data collected from this question is summarized in Table 10-19. As 
shown in Figure 6-4, the average rating for the conformance box option was 4.84 (SD=0.61, 
N=50). The average rating for the deviation scale option was 4.84 (SD=0.76, N=50). The 
average rating for no trend indication was 4.18 (SD=0.75, N=100). The results indicate a slight 
enhancement in the subject pilot’s ability to judge the aircraft’s spacing progress with either 
trend indicator. The benign nature of the recorded spacing scenario did not cause significant 
deviations to be displayed by either trend indicator, possibly diminishing the perception of its 
potential usefulness (see comments in Section 6.3.4). In response to the helpfulness question (see 
Table 10-23), the average ratings indicate a modest preference for the conformance box option 
(3.98, SD=1.19, N=49 2 ) vs. the deviation scale option (3.48, SD=1.22, N=48 2 ). 


Helpfulness 
Scale [1-5] 


■ deviation scale 


Ability to Judge Progress 
Scale (1-6) 
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Average Rating 
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■ confirmance box 

■ none 


Figure 6-4: Trend Indicator Option Average Ratings. 


6.2 Subject Pilot Ratings of Interface Importance to APS-DPA Operations 

An evaluation of the relative importance of the information conveyed by the design elements was 
obtained via the final questionnaire. The data was analyzed to verify that each element category 
was considered useful, and to further determine which characteristics of each element was 
considered the most useful. This data indicates that subject pilots considered all of the element 
categories to be important, but not equally important. 

Subject pilots were asked to rate the importance of each of the four categories of design elements 
(i.e., PDS target speed indicator, PDS mode, target aircraft depiction and trend indicator). The 
pertinent data collected from this question is summarized in Table 10-27. Using a scale of 5 
“very important” to 1 “not important”, the responses in ra nk order were: 

1 . PDS target speed indicator (average rating = 4.92, SD = 0.27, N=25); 

2. Target aircraft depiction (average rating = 4.72, SD = 0.60, N=25),; 

3. PDS mode annunciation (average rating = 4.36, SD = 0.89, N=25); 

4. Trend indicator (average rating = 4.12, SD = 1.18, N=25). 

For each design element category, subject pilots were asked to rate the importance of several 
characteristics of that element. For example, relative to PDS target speed, they were asked to rate 
the importance of being able to determine both the current target speed, and changes to the target 
speed, using a scale of 5 “very important” to 1 “not important”. The results shed light on which 
element characteristics were most valued, and provide a context for the element preferences 
discussed later. 

For the PDS target speed indicator (see Table 10-28), it is clear that both of the element 
characteristics were considered very important; subject pilots responded that detecting speed 


changes (average rating = 4.80) was almost as important as displaying the PDS-commanded 
speed (average rating = 4.92). 

For the target aircraft depiction (see Table 10-29), it is interesting to note that out of eight listed 
characteristics, the three most important (ownship relative position, active spacing aircraft and 
number of target aircraft) were shown without the use of the data block. While both of the target 
aircraft design alternatives were able to depict the ownship relative position and number of 
spacing targets, only one of the alternatives (“coupled with data block”) was implemented to also 
indicate the active spacing aircraft with a selected circle. This one highly valued characteristic 
may be a large contributor to the significant preference for this design alternative discussed later. 
The rank ordering of the five remaining characteristics (spacing deviation, aircraft ID, assigned 
runway, aircraft type, spacing interval type) indicate the infonnation within the data block 
implementation that subject pilots found most useful. 

For the PDS mode annunciation (see Table 10-30), it is interesting to note that two 
characteristics, current mode (average rating = 4.24) and changes to mode (average rating = 

4.20) were rated significantly higher than the ability to determine the next mode in sequence 
(average rating = 3.44). While both of the PDS mode design alternatives were able to show 
current mode and changes to mode, only the FMA design alternative was able to show the next 
PDS mode in sequence. This characteristic was still considered important, and may have been a 
contributor to the preference for the FMA design option (see Section 6.3.2.) 

6.3 Subject Pilot Interface Preferences 

The final questionnaire asked subject pilots to indicate a preference for individual design 
elements in each of four categories; PDS target speed; PDS mode annunciation, target aircraft 
depiction and trend indicator. Subject pilots were given a choice of both design options in each 
category, or “None” to indicate no option was preferred. They were also asked to consider 
whether their preference changed when the elements were considered in combination rather than 
individually, (only one subject pilot answered in the affirmative). The results are summarized in 
Table 10-31 through Table 10-35. 

6.3.1 PDS Target Speed 

Subject pilot responses were almost evenly split on which of the two PDS target speed display 
options was preferred, with thirteen (52%) preferring the green speed bug option and twelve 
(48%) preferring the textual speed indication. This result confirms the hypothesis that both 
options were usable and acceptable, and echoes the usability ratings discussed previously. A 
pilot who favored the green speed bug commented “. . .best presented as a speed bug for instant 
comparison with actual speed.” A pilot who preferred the text option said, “I prefer the text 
version of PS (sic) target speed because I can always find it. Rapid changes to the bug display 
can cause it to disappear.” 



6.3.2 PDS Mode Annunciation 


Subject pilot responses slightly favored the FMA integration option of PDS mode display, with 
fifteen (60%) preferring the FMA option and ten (40%) preferring the textual mode indication. 
This result is again similar to the usability and helpfulness ratings discussed previously, where 
both options were rated almost equally useful. A pilot who preferred the FMA option 
commented, “Most aircrews are trained to be very diligent on FMA observation, mode changes, 
and correct presentation for desired flight mode. So basically this would be in normal aircrew 
scan/procedures. . .” A pilot who preferred the text option said, “. . .1 think it will enhance 
understanding and avoid confusion by keeping the PDS modes in the text box and separate from 
the VNAV mode annunciators. I know at this airline that pilot understanding of the available 
VNAV modes and their annunciations is a bit of a weak point. By adding more to that portion of 
the FMA annunciations the confusion may increase.” 

6.3.3 Target Aircraft Depiction 

Subject pilot responses were overwhelmingly in favor of the “coupled with data block” option, 
with twenty-four (96%) preferring this option and one (4%) preferring the “coupled only” option. 
In examining the ratings on target aircraft characteristics discussed previously, this result can 
largely be attributed to one characteristic of the preferred option, which was the selected circle 
drawn around the active spacing aircraft. This is echoed in comments such as, “I want to know 
which aircraft is controlling my speed... the circle is a BIG HELP”, and, “I prefer the circle with a 
minimal data block adjacent to the aircraft depiction to include A/C number, type, and runway 
assignment until Final mode where the info on the ND should no longer be depicted.”. The one 
subject pilot who preferred the “coupled only” option offered, “I like the circle around it, but this 
is not needed info in most cases.” 


6.3.4 Trend Indication 

Subject pilot responses were in favor of the conformance box trend indicator display option, with 
seventeen (68%) preferring this option, five (20%) preferring the deviation scale option, and 
three (12%) preferring no trend indicator. Usability ratings for these options were equal, and the 
helpfulness rating slightly favored the confonnance box. Written comments were helpful in 
understanding the more dramatic preferences expressed in this result. Several comments were 
made calling the conformance box “intuitive”, while an equal number of comments labeled the 
deviation scale “confusing”, largely for its similarity to the VNAV path indicator on the opposite 
side of the ND. One commenter said, “My preference is Conformance Box over Deviation Scale. 
Deviation Scale looks too much like VDEV (sic) scale and could be confused. Confonnance Box 
is also along track and easy to interpret.” Of the pilots prefening the deviation scale, one 
commented, “...prefer the side deviation indicator, as changes more evident. Probably due to 
constant scale presentation.” Of the three pilots who preferred no trend indication, comments 
included “. . .provides information I have no control over. . “. . .slightly useful but not 
necessary. . and “. . .tough to judge because the deviations were so minor”. 



6.3.5 Other Preference Considerations 


The one subject pilot who changed preferences when considering the design elements in 
combination rather than individually responded with an individual preference for the FMA 
option of PDS mode annunciation, but preferred the text option when considered in combination. 
The motivation appeared to be based on ambivalence for the PDS mode annunciation options, 
coupled with a preference to keep both PDS target speed and mode displays together on the PFD. 

6.4 Subject Pilot Preferences for Combinations of Interfaces 

In the post-run questionnaire, subject pilots evaluated the presentation of each spacing scenario’s 
design elements for their overall understandability, the level of confidence they provided, and for 
their intuitiveness and user-friendliness. This data was analyzed to detennine if the results 
coincided with the individual usability ratings and preferences for the design element options, 
and to see if further insights could be gained about the effectiveness of element combinations. 

Understandability, confidence and intuitiveness were rated using a scale of 5 “very 
understandable, confident, intuitive” to 1 “not understandable, not confident, not intuitive”. The 
data collected for these questions is shown in Table 10-24 through Table 10-26. A summary of 
these results is also shown in Table 6-1, which facilitates a direct comparison of the average 
rating per run for each question, along with an overall rank order of each run’s average rating. 
The “Combined Rating” column of this table combines the weighted responses for each 
question’s responses from Table 10-24 through Table 10-26 into a single average rating. The 
runs are rank-ordered from top to bottom according to the combined rating. The design elements 
presented during each run are displayed as icons for reference. 

The results indicate a consistent ra nk ordering of each run’s average rating from top to bottom. It 
should be noted that the runs were presented in a different order for each subject pilot, so this is 
likely not the result of a bias introduced by the order of presentation. Also, since these were post- 
run evaluations, it was not a post-experiment determination of the subject pilot’s preferred 
combination of elements. Both the average ratings and the combined ratings show that runs 
which displayed either of the trend indicator options (R5, R6, R7, and R8) scored consistently 
higher than those without a trend indicator (Rl, R2, R3, and R4). Further, of the runs which 
displayed a trend indicator, those displaying the “coupled with data block” option of target 
aircraft display scored highest for both the mean responses and the combined rating (R7 and R8). 

The results seem to indicate that the design elements presented on the ND drove the rating 
criteria of understandability, confidence and intuitiveness. Subject pilots may have interpreted 
these criteria as primarily strategic in nature, and rated based on the element combination’s 
ability to provide the “big picture”. A contributing factor may have been that the design element 
categories which garnered the highest individual preference scores (target aircraft and trend 
display) preempted consideration of the other categories (PDS target speed and PDS mode), for 
which preferences were not as clearly delineated. Another possibility is that subject pilots may 
have adopted a preferred set of design elements during the pre-experiment briefing, and 
answered the post-run evaluations accordingly. 



Table 6-1 : Combined Design Element Preferences. 
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6.5 Suggested Improvements by Subject Pilots 
6.5.1 PDS Target Speed Interface 

Several written comments suggested that a combination of the green speed bug and textual 
modes would be better, and that both options of PDS target speed display could be useful, 
depending on the instrument scan pattern demanded by a particular phase of flight. A mockup of 
one subject pilot’s suggestion for the combination of the two options is shown in Figure 6-5, in 
which the textual presentation of PDS target speed ha been relocated to the area above the speed 
tape, juxtaposing it with the commanded speed text. This concept facilitates easy comparison of 
the two speeds. 






















It was also suggested by some that, while generally important during APS-DPA operations, PDS 
target speed (as well as other PDS design elements) may be an unnecessary distraction during 
final approach, when spacing deviations are no longer being countered by PDS target speed. 


Textual 

(Suggested) 


Graphic 



Figure 6-5: Suggested Concept for PDS Target Speed Display. 


6.5.2 PDS Mode Annunciation 

Several subject pilots commented that the addition of the PDS information to the FMA was 
“confusing”. Some implementation anomalies in the ASTOR depiction of this option may have 
contributed to the confusion (see Appendix E: ASTOR Implementation Issues). However, 
several subject pilots commented that the FMA information is vital to understanding the 
operation of modern Boeing automation, and therefore this was the rightful place for PDS mode 
annunciation. In retrospect, it may have been less confusing if, as some suggested, the PDS 
annunciation were added by introducing a new segment to the FMA. This would have had the 
advantage of not changing the pilot’s understanding of any existing segment, and would have 
isolated any additional training to that required for the new PDS segment. 

6.5.3 Target Aircraft Depiction 

In examining the comments made, it is clear that the identification of the active spacing aircraft 
with a select circle was perceived as enhancing the situation awareness during the spacing 
operation. There were several suggested improvements to enhancing the awareness of the active 
spacing aircraft in the event of a change of active aircraft. These suggestions included: 




1) “blinking” of the circle, 2) changing the color of the circle, and 3) changing the text color of 
the ID of the active spacing aircraft to magenta. 

Several comments were made regarding the efficacy of the data block. Comments supporting the 
data block information were offset by comments recommending: 1) less information in the data 
block, 2) allowing the data block to be selectively turned on or off, and, 3) automatically turning 
the data block off on final approach to eliminate it as a distraction. It was also noted that the data 
block as implemented could conflict with the display of Navigation Aid (NAVAID) information, 
which is selected via the EFIS control panel. 

6.5.4 Trend Indication 

Several comments were made about the deviation scale option of the trend indicator being 
confusing. Most cited the scales similarity to the VNAV deviation scale on the opposite side of 
the ND, which would also be displayed while flying a CDA profile. No suggestions for 
improvement were made by the commenters, but the essence of the comments were used to 
propose a mockup of an alternative deviation scale shown in Figure 6-6. In this concept, the scale 
pointer indicating the relative spacing deviation from nominal has been replaced with a small, 
green ownship-like triangle indicator. The original sense of the scale has been reversed, such that 
negative deviations are up or forward on the scale, and positive deviations are down or aft, 
improving the scale’s appearance as a longitudinal deviation indicator. 



Figure 6-6: Alternate Trend Indicator Concept. 




7 Conclusions 


This study built upon previous experiments focused on the Airborne Precision Spacing concept. 
Several candidate interfaces were designed to support spacing operations for aircraft perfonning 
CDAs while also managing assigned spacing intervals from two other arriving aircraft on 
dependent parallel approaches. An experiment was designed to verify the usability of these 
interfaces using recorded spacing scenarios, which were implemented using an existing 
procedure simulator (ASTOR). 

The results of the study indicate that several options exist for usable interfaces to support these 
spacing operations. All of the interface options evaluated by subject pilots were considered 
acceptable, and valuable insight was collected on their relative effectiveness and importance to 
the conduct of spacing operations. In particular, the key results indicate the following: 

1 . both textual and graphical presentations of pds target speed were considered useful; 

2. target aircraft indication is considered especially useful if it also indicates which of the 
two targets is the active spacing aircraft; 

3. subject pilots favored the flight mode annunciator option over the text option for pds 
mode annunciation, which should be considered carefully since it would likely present a 
more significant pilot training issue than the text option; and 

4. spacing trend indication was considered the least important design element, yet it was 
also considered useful by most subject pilots. 

The results also highlighted the need for additional investigation. The evaluations of options for 
presenting PDS target speed indicated equal preference, and suggestions to display both on the 
PFD (similar to the commanded speed) have merit. A more thorough consideration of the 
integration of PDS annunciation into the FMA is recommended, as most subject pilots preferred 
this for monitoring autopilot controlled modes of operation, and there were issues noted with the 
study’s implementation. The actual benefit of the data block containing target aircraft 
infonnation may have been masked by its association with the active spacing aircraft select 
circle, but sufficient support existed for the refinement of data block information to be 
investigated. Future studies should characterize the behavior of the trend indicators using a 
variety of spacing scenarios, including scenarios with less benign conditions than used in this 
experiment. The responses for the trend indicator showed it to be lowest in relative importance, 
yet a trend indicator was always included in preferred combinations of elements. This result 
warrants further investigation, especially with off-nominal scenarios, of whether a trend 
indicator is imparting useful information during a spacing operation, or if off-nominal alerts of 
another form (EICAS messages, audio alerts, etc.) would be sufficient. 

Finally, the ASTOR implementation used to create the spacing scenarios leaves a legacy of 
usable software which can easily be used to support future studies or demonstrations. Many of 
the discussed suggestions for improvement of the interfaces involve simple software 
development and/or “bug” fixes. 
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10 Appendix A: Tabulated Experimental Results 

10.1 Results of Post-Run Questionnaires 

This is the response data from the post-run questionnaire for the eight (8) test runs Rl- 
R8. This questionnaire was structured to gather responses regarding design element 
usability and acceptability (i.e., were they understandable, helpful, intuitive, confidence- 
inspiring). Questions were asked which explored the usability of individual design 
elements and their characteristics, and also for combinations of elements. Except for 
solicited additional comments, responses were collected using Likert-type scales. 

The text of each question precedes a set of tables showing the results. The data collected 
for each question is tabulated to make it easier to compare the results from individual test 
runs. Refer to Table 10-1 for a matrix which shows which display elements were 
presented to the subject during each run. Table data is organized by grouping runs which 
displayed a particular option. A summary entry identifies the design option, the overall 
ratings and sample size. Note that the “Rating Average” is the weighted average of the 
responses based on the values shown in the “Weight” row in the table, the “SD” is the 
population standard deviation, and “N” is the sample size, which is the number of valid 
responses collected for the particular run. 


Table 10-1: Test Run Matrix Showing Display Options Enabled 


Run 

PDS Target Speed 
Option 

PDS Mode 
Option 

Target Aircraft 
Option 

Trend Indicator 
Option 

R1 

Text 

Text 

Coupled Only 

None 

R2 

Text 

FMA 

Coupled Only 

None 

R3 

Speed Bug 

Text 

Coupled Only 

None 

R4 

Speed Bug 

FMA 

Coupled Only 

None 

R5 

Text 

Text 

Coupled Only 

Conformance Box 

R6 

Text 

Text 

Coupled Only 

Deviation Scale 

R7 

Text 

Text 

Coupled w/Data 

Conformance Box 

R8 

Text 

Text 

Coupled w/Data 

Deviation Scale 


10.1.1 Usability of Display Elements 

The following post-run questionnaire responses rated the subject’s ability to understand 
the infonnation being presented by the design elements, which was considered a key 
component of usability. Questions about PDS target speed, PDS mode annunciation and 
target aircraft depiction were sub-divided according to characteristics of the infonnation 
presented, i.e., for PDS target speed, both current target speed and changes to target 
speed were rated. The runs are grouped by the design element option presented during the 
run. An overall rating for the design element option is shown in a summary line of the 
table. The results are discussed in more detail in 6. 1 . 





“Based on what you experienced during the scenario you just completed, please rate your 
ability to identify the following infonnation regarding the spacing operation.” 

10.1.1.1 PDS Target Speed 


Table 10-2: Current Target Speed 


Run 

Very 

Easy 

Easy 

Identi- 

fiable 

Not 

Easy 

Difficult 

Not 

Able 

N/A 

Rating 

Avg. 

N 

SD 

Wt. 

(6) 

(5) 

(4) 


(3) 


(2) 


(1) 






Text Option Ratinqs 
R1 11 12 


2 


0 


0 

0 


0 

5.36 

25 

0.62 

R2 

13 

9 


3 


0 


0 

0 


0 

5.40 

25 

0.69 

R5 

10 

13 


2 


0 


0 

0 


0 

5.32 

25 

0.61 

R6 

13 

10 


2 


0 


0 

0 


0 

5.44 

25 

0.64 

R7 

14 

8 


3 


0 


0 

0 


0 

5.44 

25 

0.70 

R8 

13 

10 


2 


0 


0 

0 


0 

5.44 

25 

0.64 

Total 

58 

60 


31 


1 


0 

0 


0 

5.40 

150 

0.65 

Speed Bug Option Ratinqs 
R3 12 9 3 

0 


1 


0 

0 


5.24 

25 

0.95 

R4 

12 

7 

5 


1 


0 


0 

0 


5.20 

25 

0.89 

Total 

24 

15 

9 


0 


1 


1 

0 


5.22 

50 

0.92 


Table 10-3: Changes to Target Speed 


Run 

Very 

Easy 

Easy 

Identi- 

fiable 

Not 

Easy 

Difficult 

Able 

Rating 

N/A Avg. 

N 

SD 

1 Wt. 

(6) 

(5) 

(4) 

(3) 

(2) 

(1) 





Text Option Ratinqs 









R1 

7 

8 

10 

0 

0 

0 

0 

4.88 

25 

0.82 

R2 

10 

6 

8 

1 

0 

0 

0 

5.00 

25 

0.94 

R5 

7 

11 

6 

1 

0 

0 

0 

4.96 

25 

0.82 

R6 

8 

9 

7 

1 

0 

0 

0 

4.96 

25 

0.87 

R7 

10 

9 

6 

0 

0 

0 

0 

5.16 

25 

0.78 

R8 

12 

8 

3 

2 

0 

0 

0 

5.20 

25 

0.94 

Total 

74 

62 

14 

0 

0 

0 

0 

5.40 

150 

0.65 

Speed Buq Option Ratinqs 








R3 

10 

9 

5 

0 

1 

0 

0 

5.08 

25 

0.98 

R4 

10 

7 

5 

2 

1 

0 

0 

4.92 

25 

1.13 

Total 

20 

16 

10 

2 

2 

0 

0 

5.00 

50 

1.06 




10.1.1.2 PDS Mode 


Table 10-4: Current Mode 


Run 

Very 

Easy 

Easy 

Identi- 

fiable 

Not 

Easy 

Difficult 

Not 

Able 

N/A 

Rating 

Avg. 

N 

SD 

Wt. 

(6) 

(5) 

(4) 

(3) 

(2) 

(1) 





Text Option Ratinqs 
R1 8 12 

5 

0 

0 

0 

0 

5.12 

25 

0.71 

R3 

8 

8 

9 

0 

0 

0 

0 

4.96 

25 

0.82 

R5 

10 

13 

2 

0 

0 

0 

0 

5.32 

25 

0.61 

R6 

11 

10 

3 

1 

0 

0 

0 

5.24 

25 

0.81 

R7 

9 

9 

7 

0 

0 

0 

0 

5.08 

25 

0.80 

R8 

12 

8 

5 

0 

0 

0 

0 

5.28 

25 

0.78 

Total 

58 

60 

31 

1 

0 

0 

0 

5.17 

150 

0.77 

FMA Option Ratinqs 
R2 12 9 

4 

0 

0 

0 

0 

5.32 

25 

0.73 

R4 

12 

6 

5 

0 

1 

1 

0 

5.00 

25 

1.30 

Total 

24 

15 

9 

0 

1 

1 

0 

5.16 

50 

1.07 


Table 10-5: Changes to Mode 


Run 

Very 

Easy 

Easy 

Identi- 

fiable 

Not 

Easy 

Difficult 

Not 

Able 

N/A 

Rating 

Avg. 

N 

SD 

1 Wt. 

(6) 

(5) 

(4) 

(3) 

(2) 

(1) 

(0) 




Text Option Ratinqs 
R1 5 10 

6 

3 

0 

0 

1 

4.52 

24 

0.93 

R3 

5 

8 

10 

2 

0 

0 

0 

4.64 

25 

0.89 

R5 

7 

9 

9 

0 

0 

0 

0 

4.92 

25 

0.80 

R6 

7 

10 

7 

1 

0 

0 

0 

4.92 

25 

0.84 

R7 

7 

9 

9 

0 

0 

0 

0 

4.92 

25 

0.80 

R8 

8 

8 

7 

2 

0 

0 

0 

4.88 

25 

0.95 

Total 

39 

54 

48 

8 

0 

0 

1 

4.83 

149 

0.88 

FMA Option Ratinqs 
R2 9 10 

6 

0 

0 

0 

0 

5.12 

25 

0.77 

R4 

9 

8 

5 

1 

1 

1 

0 

4.80 

25 

1.30 

Total 

18 

18 

11 

1 

1 

1 

0 

4.96 

50 

1.08 
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Table 10-6: Next Mode in Sequence 


Run 

Very 

Easy 

Easy 

Identi- 

fiable 

Not 

Easy 

Difficult 

Not 

Able 

N/A 

Rating 

Avg. 

N 

SD 

Wt. 

(6) 

(5) 

(4) 

(3) 

(2) 

(1) 





Text Option Ratings 









R1 

3 

3 

3 

1 

0 

10 

5 

2.90 

20 

2.02 

R3 

3 

1 

5 

3 

1 

8 

4 

2.95 

21 

1.81 

R5 

5 

2 

3 

1 

0 

8 

6 

3.32 

19 

2.13 

R6 

4 

6 

2 

2 

0 

5 

6 

3.84 

19 

1.90 

R7 

4 

4 

2 

0 

0 

11 

4 

3.00 

21 

2.16 

R8 

4 

3 

4 

2 

0 

7 

5 

3.40 

20 

1.96 

Total 

23 

19 

19 

9 

1 

49 

30 

3.23 

120 

2.03 

FMA Option Ratings 









R2 

10 

9 

1 

1 

0 

3 

1 

4.79 

24 

1.61 

R4 

7 

11 

1 

2 

0 

1 

3 

4.91 

22 

1.20 

Total 

17 

20 

2 

3 

0 

4 

4 

4.85 

46 

1.43 


10.1.1.3 Target Aircraft: 


Table 10-7: How Many (one or two) 


Run 

Very 

Easy 

Easy 

Identi- 

fiable 

Not 

Easy 

Difficult 

Not 

Able 

N/A 

Rating 

Avg. 

N 

SD 

1 Wt. 

(6) 

(5) 

(4) 

(3) 

(2) 

(1) 





Coupled Only Option Ratings 








R0 

2 

4 

2 

0 

0 

0 

17 

5.00 

8 

0.71 

R1 

7 

15 

3 

0 

0 

0 

0 

5.16 

25 

0.61 

R2 

9 

12 

4 

0 

0 

0 

0 

5.20 

25 

0.69 

R3 

9 

13 

3 

0 

0 

0 

0 

5.24 

25 

0.65 

R4 

9 

14 

2 

0 

0 

0 

0 

5.28 

25 

0.60 

R5 

12 

11 

2 

0 

0 

0 

0 

5.40 

25 

0.63 

Total 

58 

73 

19 

0 

0 

0 

0 

5.26 

150 

0.67 

Coupled w/ Data Block Option Ratings 






R7 

13 

11 

1 

0 

0 

0 

0 

5.48 

25 

0.57 

R8 

16 

9 

0 

0 

0 

0 

0 

5.64 

25 

0.48 

Total 

29 

20 

1 

0 

0 

0 

0 

5.56 

50 

0.54 
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Table 10-8: Active Spacing Aircraft 


Run 

Very 

Easy 

Easy 

Identi- 

fiable 

Not 

Easy 

Difficult 

Not 

Able 

N/A 

Rating 

Avg. 

N 

SD 

Wt. 

(6) 

(5) 

(4) 

(3) 

(2) 

(1) 





Coupled Only Option Ratings 








R1 

2 

5 

6 

8 

2 

1 

1 

3.75 

24 

1.23 

R2 

3 

6 

10 

3 

2 

1 

0 

4.08 

25 

1.23 

R3 

4 

7 

8 

6 

0 

0 

0 

4.36 

25 

1.02 

R4 

3 

7 

11 

4 

0 

0 

0 

4.36 

25 

0.89 

R5 

4 

8 

6 

6 

0 

1 

0 

4.28 

25 

1.22 

R6 

2 

6 

10 

6 

0 

1 

0 

4.04 

25 

1.08 

Total 

18 

39 

51 

33 

4 

4 

1 

4.15 

149 

1.14 

Coupled w/ Data Block Option Ratings 






R7 

20 

4 

1 

0 

0 

0 

0 

5.76 

25 

0.51 

R8 

20 

4 

0 

1 

0 

0 

0 

5.72 

25 

0.66 

Total 

40 

8 

1 

1 

0 

0 

0 

5.74 

50 

0.59 


Table 10-9: Ownship Position Relative to Spacing Aircraft 


Run 

Very 

Easy 

Easy 

Identi- 

fiable 

Not 

Easy 

Difficult 

Not 

Able 

N/A 

Rating 

Avg. 

N 

SD 

1 Wt. 

(6) 

(5) 

(4) 

(3) 

(2) 

(1) 





Coupled Only Option Ratings 








R1 

5 

8 

8 

1 

2 

1 

0 

4.40 

25 

1.30 

R2 

4 

8 

9 

3 

1 

0 

0 

4.44 

25 

1.02 

R3 

8 

4 

8 

4 

1 

0 

0 

4.56 

25 

1.20 

R4 

5 

9 

6 

4 

1 

0 

0 

4.52 

25 

1.10 

R5 

9 

7 

5 

2 

2 

0 

0 

4.76 

25 

1.24 

R6 

4 

10 

10 

1 

0 

0 

0 

4.68 

25 

0.79 

Total 

35 

46 

46 

15 

7 

1 

0 

4.56 

150 

1.13 

Coupled w/ Data Block Option Ratings 






R7 

17 

7 

1 

0 

0 

0 

0 

5.64 

25 

0.56 

R8 

17 

6 

2 

0 

0 

0 

0 

5.60 

25 

0.63 

Total 

34 

13 

3 

0 

0 

0 

0 

5.62 

50 

0.60 
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Table 10-10: Aircraft ID(s) 


Run 

Very 

Easy 

Easy 

Identi- 

fiable 

Not 

Easy 

Difficult 

Not 

Able 

N/A 

Rating 

Avg. 

N 

SD 

Wt. 

(6) 

(5) 

(4) 

(3) 

(2) 

(1) 





Coupled Only Option Ratings 








R1 

6 

5 

9 

2 

1 

2 

0 

4.28 

25 

1.43 

R2 

4 

11 

7 

1 

1 

1 

0 

4.52 

25 

1.17 

R3 

7 

8 

6 

2 

1 

1 

0 

4.60 

25 

1.30 

R4 

2 

9 

11 

0 

1 

2 

0 

4.20 

25 

1.23 

R5 

6 

6 

10 

2 

1 

0 

0 

4.56 

25 

1.06 

R6 

3 

8 

11 

1 

1 

1 

0 

4.32 

25 

1.12 

Total 

28 

47 

54 

8 

6 

7 

0 

4.41 

150 

1.23 

Coupled w/ Data Block Option Ratings 






R7 

15 

7 

3 

0 

0 

0 

0 

5.48 

25 

0.70 

R8 

18 

6 

1 

0 

0 

0 

0 

5.68 

25 

0.55 

Total 

33 

13 

4 

0 

0 

0 

0 

5.58 

50 

0.64 


Table 10-11: Spacing Interval Type: NASA 893 


Run 

Very 

Easy 

Easy 

Identi- 

fiable 

Not 

Easy 

Difficult 

Not 

Able 

N/A 

Rating 

Avg. 

N 

SD 

1 Wt. 

(6) 

(5) 

(4) 

(3) 

(2) 

(1) 





Coupled Only Option Ratings 








R1 

2 

5 

13 

3 

1 

1 

0 

4.04 

25 

1.08 

R2 

4 

2 

12 

4 

3 

0 

0 

4.00 

25 

1.17 

R3 

2 

7 

13 

1 

1 

1 

0 

4.20 

25 

1.06 

R4 

0 

6 

15 

1 

1 

2 

0 

3.88 

25 

1.07 

R5 

2 

7 

8 

6 

2 

0 

0 

4.04 

25 

1.08 

R6 

1 

8 

11 

3 

1 

1 

0 

4.08 

25 

1.06 

Total 

11 

35 

72 

18 

9 

5 

0 

4.04 

150 

1.09 

Coupled w/ Data Block Option Ratings 






R7 

9 

11 

4 

1 

0 

0 

0 

5.12 

25 

0.82 

R8 

7 

13 

4 

1 

0 

0 

0 

5.04 

25 

0.77 

Total 

16 

24 

8 

2 

0 

0 

0 

5.08 

50 

0.80 
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Table 10-12: Spacing Interval Type: NASA 859 


Run 

Very 

Easy 

Easy 

Identi- 

fiable 

Not 

Easy 

Difficult 

Able 

Rating 

N/A Avg. 

N 

SD 

Wt. 

(6) 

(5) 

(4) 

(3) 

(2) 

(1) 





Coupled Only Option Ratings 







R1 

1 

4 

14 

3 

1 

2 

0 

3.80 

25 

1.13 

R2 

4 

2 

12 

3 

4 

0 

0 

3.96 

25 

1.22 

R3 

2 

7 

13 

1 

1 

1 

0 

4.20 

25 

1.06 

R4 

0 

6 

15 

1 

1 

2 

0 

3.88 

25 

1.07 

R5 

2 

7 

8 

5 

2 

1 

0 

3.96 

25 

1.22 

R6 

1 

6 

13 

3 

1 

1 

0 

4.00 

25 

1.02 

Total 

10 

32 

75 

16 

10 

7 

0 

3.97 

150 

1.13 

Coupled w / Data Block Option Ratings 






R7 

8 

10 

6 

1 

0 

0 

0 

5.00 

25 

0.85 

R8 

7 

11 

6 

1 

0 

0 

0 

4.96 

25 

0.82 

Total 

15 

21 

12 

2 

0 

0 

0 

4.98 

50 

0.84 


Table 10-13: Spacing Deviation: NASA 893 


Run 

Very 

Easy 

Easy 

Identi- 

fiable 

Not 

Easy 

Difficult 

Not 

Able 

N/A 

Rating 

Avg. 

N 

SD 

1 Wt. 

(6) 

(5) 

(4) 

(3) 

(2) 

(1) 





Coupled Only Option Ratings 








R1 

0 

3 

14 

4 

3 

1 

0 

3.60 

25 

0.98 

R2 

1 

3 

11 

3 

6 

1 

0 

3.48 

25 

1.20 

R3 

1 

5 

10 

5 

2 

2 

0 

3.68 

25 

1.22 

R4 

0 

5 

14 

3 

3 

0 

0 

3.84 

25 

0.88 

R5 

3 

7 

11 

1 

3 

0 

0 

4.24 

25 

1.11 

R6 

1 

9 

8 

4 

2 

1 

0 

4.00 

25 

1.17 

Total 

6 

32 

68 

20 

19 

5 

0 

3.81 

150 

1.13 

Coupled w / Data Block Option Ratings 






R7 

11 

9 

4 

1 

0 

0 

0 

5.20 

25 

0.85 

R8 

5 

12 

6 

1 

1 

0 

0 

4.76 

25 

0.95 

Total 

16 

21 

10 

2 

1 

0 

0 

4.98 

50 

0.93 
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Table 10-14: Spacing Deviation: NASA 859 


Run 

Very 

Easy 

Easy 

Identi- 

fiable 

Not 

Easy 

Difficult 

Not 

Able 

N/A 

Rating 

Avg. 

N 

SD 

Wt. 

(6) 

(5) 

(4) 

(3) 

(2) 

(1) 





Coupled Only Option Ratings 








R1 

0 

3 

14 

4 

3 

1 

0 

3.60 

25 

0.98 

R2 

1 

3 

12 

3 

5 

1 

0 

3.56 

25 

1.17 

R3 

1 

5 

10 

5 

2 

2 

0 

3.68 

25 

1.22 

R4 

0 

5 

13 

3 

3 

1 

0 

3.72 

25 

1.04 

R5 

3 

6 

10 

2 

3 

1 

0 

4.04 

25 

1.28 

R6 

1 

8 

10 

4 

2 

0 

0 

4.08 

25 

0.98 

Total 

6 

30 

69 

21 

18 

6 

0 

3.78 

150 

1.14 

Coupled w/ Data Block Option Ratings 






R7 

9 

9 

5 

1 

1 

0 

0 

4.96 

25 

1.04 

R8 

5 

10 

7 

2 

1 

0 

0 

4.64 

25 

1.02 

Total 

14 

19 

12 

3 

2 

0 

0 

4.80 

50 

1.04 


Table 10-15: Assigned Runway: NASA 893 


Run 

Very 

Easy 

Easy 

Identi- 

fiable 

Not 

Easy 

Difficult 

Able 

Rating 

N/A Avg. 

N 

SD 

1 Wt. 

(6) 

(5) 

(4) 

(3) 

(2) 

(1) 





Coupled Only Option Ratings 







R1 

1 

1 

8 

5 

4 

5 

1 

2.96 

24 

1.37 

R2 

1 

4 

3 

8 

3 

5 

1 

3.04 

24 

1.46 

R3 

2 

1 

4 

7 

4 

6 

1 

2.83 

24 

1.49 

R4 

1 

1 

8 

5 

3 

6 

1 

2.92 

24 

1.41 

R5 

2 

1 

7 

7 

2 

5 

1 

3.13 

24 

1.45 

R6 

1 

5 

2 

8 

3 

4 

2 

3.17 

23 

1.46 

Total 

8 

13 

32 

40 

19 

31 

7 

3.01 

143 

1.45 

Coupled w / Data Block Option Ratings 






R7 

12 

9 

3 

0 

0 

1 

0 

5.20 

25 

1.10 

R8 

12 

8 

2 

3 

0 

0 

0 

5.16 

25 

1.01 

Total 

24 

17 

5 

3 

0 

1 

0 

5.18 

50 

1.05 
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Table 10-16: Assigned Runway: NASA 859 


Run 

Very 

Easy 

Easy Identi- 
fiable 

Not 

Easy 

Difficult 

Able 

Rating 
N/A Avg. 

N 

SD 

Wt. 

(6) 

(5) 

(4) 

(3) 

(2) 

(1) 





Coupled Only Option Ratings 







R1 

0 

1 

6 

5 

5 

7 

1 

2.54 

24 

1.26 

R2 

1 

3 

2 

9 

3 

6 

1 

2.83 

24 

1.43 

R3 

1 

1 

4 

7 

4 

7 

1 

2.63 

24 

1.38 

R4 

0 

1 

5 

7 

4 

7 

1 

2.54 

24 

1.22 

R5 

0 

1 

4 

10 

2 

7 

1 

2.58 

24 

1.19 

R6 

1 

1 

2 

10 

3 

6 

2 

2.65 

23 

1.31 

Total 

3 

8 

23 

48 

21 

40 

7 

2.63 

143 

1.30 

Coupled w/ Data Block Option Ratings 






R7 

10 

10 

1 

3 

0 

1 

0 

4.96 

25 

1.25 

R8 

11 

7 

3 

2 

1 

1 

0 

4.88 

25 

1.37 

Total 

21 

17 

4 

5 

1 

2 

0 

4.92 

50 

1.31 


Table 10-17: Aircraft Type (i.e., 757, hvy, etc.): NASA 893 


Run 

Very 

Easy 

Easy 

Identi- 

fiable 

Not 

Easy 

Difficult 

Not 

Able 

N/A 

Rating 

Avg. 

N 

SD 

1 Wt. 

(6) 

(5) 

(4) 

(3) 

(2) 

(1) 





Coupled Only Option Ratings 








R1 

0 

1 

2 

7 

1 

10 

4 

2.19 

21 

1.26 

R2 

0 

0 

2 

7 

4 

9 

3 

2.09 

22 

1.04 

R3 

0 

0 

2 

6 

0 

14 

3 

1.82 

22 

1.11 

R4 

0 

0 

2 

4 

2 

14 

3 

1.73 

22 

1.05 

R5 

0 

1 

2 

7 

1 

11 

3 

2.14 

22 

1.25 

R6 

0 

1 

1 

8 

0 

10 

5 

2.15 

20 

1.24 

Total 

0 

3 

11 

39 

8 

68 

21 

2.02 

129 

1.17 

Coupled w/ Data Block Option Ratings 






R7 

13 

8 

2 

1 

0 

1 

0 

5.20 

25 

1.17 

R8 

13 

7 

3 

1 

0 

1 

0 

5.16 

25 

1.19 

Total 

26 

15 

5 

2 

0 

2 

0 

5.18 

50 

1.18 
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Table 10-18 Aircraft Type (i.e., 757, hvy, etc.): NASA 859 


Run 

Very 

Easy 

Easy Identi- 
fiable 

Not 

Easy 

Difficult 

Able 

N/A 

Rating 

Avg. 

N 

SD 

Wt. 

(6) 

(5) 

(4) 

(3) 

(2) 

(1) 





Coupled Only Option Ratings 







R1 

0 

0 

2 

6 

3 

10 

4 

2.00 

21 

1.07 

R2 

0 

0 

2 

7 

4 

9 

3 

2.09 

22 

1.04 

R3 

0 

0 

2 

6 

0 

14 

3 

1.82 

22 

1.11 

R4 

0 

0 

2 

4 

2 

14 

3 

1.73 

22 

1.05 

R5 

0 

1 

1 

8 

1 

11 

3 

2.09 

22 

1.20 

R6 

0 

0 

1 

9 

0 

10 

5 

2.05 

20 

1.07 

Total 

0 

1 

10 

40 

10 

68 

21 

1.96 

129 

1.10 

Coupled w/ Data Block Option Ratings 






R7 

11 

8 

2 

3 

0 

1 

0 

4.96 

25 

1.28 

R8 

11 

6 

4 

1 

0 

3 

0 

4.72 

25 

1.61 

Total 

22 

14 

6 

4 

0 

4 

0 

4.84 

50 

1.46 
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10.1.1.4 Trend Indication 


“Based on what you experienced during the scenario you just completed, please rate your 
ability to determine and understand the aircraft’s progress toward achieving the 
assigned spacing goal.” 


Table 10-19: Trend Indication 


Run 

Very 

Easy 

to 

Under- 

stand 

Easy 

To 

Under- 

stand 

Under- 

stand- 

able 

Not 

Easy 

to 

Under- 

stand 

Difficult 

to 

Under- 

stand 

Not 

Able 

to 

Under 

stand 

Rating 

Avg 

N 

SD 

Wt. 

(6) 

(5) 

(4) 

(3) 

(2) 

(1) 




No trend indicator ratings 








Rl 

1 

5 

14 

5 

0 

0 

4.08 

25 

0.74 

R2 

1 

5 

17 

1 

1 

0 

4.16 

25 

0.73 

R3 

1 

7 

14 

2 

1 

0 

4.20 

25 

0.80 

R4 

1 

7 

16 

0 

1 

0 

4.28 

25 

0.72 

Total 

4 

24 

61 

8 

3 

0 

4.18 

100 

0.75 

Conformance Box option ratings 







R5 

0 

15 

10 

0 

0 

0 

4.60 

25 

0.49 

R7 

6 

15 

4 

0 

0 

0 

5.08 

25 

0.63 

Total 

6 

30 

14 

0 

0 

0 

4.84 

50 

0.61 

Deviation Scale option ratings 







R6 

0 

14 

9 

2 

0 

0 

4.48 

25 

0.64 

R8 

9 

12 

4 

0 

0 

0 

5.20 

25 

0.69 

Total 

9 

26 

13 

2 

0 

0 

4.84 

50 

0.76 


The following post-run questionnaire responses rated the subject’s opinion on the 
helpfulness of the information being presented by the design element, which was 
considered a key component of usability. The runs are grouped by the design element 
option presented during the run. An overall rating for the design element option is shown 
in a summary line of the table. Note that a trend indicator was not displayed in runs Rl, 
R2, R3, and R4, so the results are not shown. The results are discussed in more detail in 
6 . 1 . 

“Using the rating scales, please rate the helpfulness of each element in understanding 
the spacing operation:” 



Table 10-20: PDS Target Speed Indicator Helpfulness 


Run 

Very 

Help- 

ful 

Help- 

ful 

Mod. 

Help- 

ful 

Little 

Help 

Not 

Help- 

ful 

Avg. 

Rating 

N 

SD 

Wt. 

(5) 

(4) 

(3) 

(2) 

(1) 




Text option ratings 







R1 

9 

12 

4 

0 

0 

4.20 

25 

0.69 

R2 

8 

13 

4 

0 

0 

4.16 

25 

0.67 

R5 

6 

18 

1 

0 

0 

4.20 

25 

0.49 

R6 

8 

12 

3 

0 

0 

4.22 

23 

0.66 

R7 

8 

15 

1 

0 

0 

4.29 

24 

0.54 

R8 

12 

9 

3 

1 

0 

4.28 

25 

0.83 

Total 

51 

79 

16 

1 

0 

4.22 

147 

0.66 

Speed Bug Option Ratings 






R3 

10 

6 

7 

0 

1 

4.00 

24 

1.04 

R4 

11 

9 

3 

2 

0 

4.16 

25 

0.92 

Total 

21 

15 

10 

2 

1 

4.08 

49 

0.99 


Table 10-21: PDS Mode Annunciation Helpfulness 


Run 

Very 

Help- 

ful 

Help- 

ful 

Mod. 

Help- 

ful 

Little 

Help 

Not 

Help- 

ful 

Avg. 

Rating 

N 

SD 

1 Wt. 

(5) 

(4) 

(3) 

(2) 

(1) 




Text option ratings 







R1 

5 

13 

7 

0 

0 

3.92 

25 

0.69 

R3 

4 

11 

7 

2 

0 

3.71 

24 

0.84 

R5 

7 

14 

4 

0 

0 

4.12 

25 

0.65 

R6 

6 

13 

4 

0 

0 

4.09 

23 

0.65 

R7 

5 

16 

2 

1 

0 

4.04 

24 

0.68 

R8 

7 

15 

2 

1 

0 

4.12 

25 

0.71 

Total 

34 

82 

26 

4 

0 

4.00 

146 

0.72 

FMA Option Ratings 







R2 

8 

10 

5 

2 

0 

3.96 

25 

0.92 

R4 

8 

12 

4 

0 

1 

4.04 

25 

0.92 

Total 

16 

22 

9 

2 

1 

4.00 

50 

0.92 




Table 10-22: Target Aircraft Indicator Helpfulness 


Run 

Very 

Help- 

ful 

Help- 

ful 

Mod. 

Help- 

ful 

Little 

Help 

Not 

Help- 

ful 

Avg. 

Rating 

N 

SD 

Wt. 

(5) 

(4) 

(3) 

(2) 

(1) 




Coupled Only Option Ratings 






R1 

5 

15 

2 

3 

0 

3.88 

25 

0.86 

R2 

4 

15 

3 

3 

0 

3.80 

25 

0.85 

R3 

2 

15 

4 

3 

0 

3.67 

24 

0.80 

R4 

8 

12 

3 

2 

0 

4.04 

25 

0.87 

R5 

4 

18 

3 

0 

0 

4.04 

25 

0.53 

R6 

6 

15 

1 

1 

0 

4.13 

23 

0.68 

Total 

29 

90 

16 

12 

0 

3.93 

147 

0.79 

Coupled w/Data Block Option Ratings 





R7 

18 

6 

0 

0 

0 

4.75 

24 

0.43 

R8 

23 

1 

1 

0 

0 

4.88 

25 

0.43 

Total 

41 

7 

1 

0 

0 

4.82 

49 

0.44 


Table 10-23: Trend Indicator Helpfulness 


Run 

Very 

Help- 

ful 

Help- 

ful 

Mod. 

Help- 

ful 

Little 

Help 

Not 

Help- 

ful 

Avg. 

Rating 

N 

SD 

1 Wt. 

(5) 

(4) 

(3) 

(2) 

(1) 




Conformance Box Option Ratings 





R5 

9 

7 

5 

1 

3 

3.72 

25 

1.31 

R7 

12 

8 

3 

0 

1 

4.25 

24 

0.97 

Total 

21 

15 

8 

1 

4 

3.98 

49 

1.19 

Deviation Scale Option Ratings 





R6 

4 

8 

5 

3 

3 

3.30 

23 

1.27 

R8 

7 

8 

5 

4 

1 

3.64 

25 

1.16 

Total 

11 

16 

10 

7 

4 

3.48 

48 

1.22 


10.1.2 Preference for Combinations of Display Elements 

The following post-run questionnaire responses rate the subject’s preferences for 
combinations of the design elements presented in the categories of overall understanding, 
confidence and intuitiveness. The tabulated results, shown in rank order of rating average 
each run, illustrate which combinations of elements the subjects preferred in each 
category. The significance of the shaded areas pertains to significant differentiations in 
the elements enabled during the run (see Table 10-1). The gray shaded runs significant 
differentiation from the other runs is that they displayed both the “Coupled w/ Data 
Block” target aircraft depiction and one of the trend indicators (confonnance box or 




deviation scale). The tan shaded runs significant differentiation from the other runs is that 
they displayed both the “Coupled Only” target aircraft depiction and one of the trend 
indicators (conformance box or deviation scale). The unshaded runs significant 
differentiation is that they displayed no trend indicator. This differentiation is also 
illustrated in Table 6-1, and preferred combinations of results are also discussed in 6.4 
(Subject Pilot Preferences for Combinations of Interfaces). 

“Please rate your overall understanding of what was happening during the APS-DPA 
operation.” 


Table 10-24: Overall Understanding 


Run 

Very 
Easy to 
Under- 
stand 

Easy to 
Under- 
stand 

Able to 
Under- 
stand 

Not 
Easy to 
Under- 
stand 

Difficult 

to 

Under- 

stand 

Not 
Able to 
Under- 
stand 

Rating 

Avg. 

N 

SD 

1 Wt. 

(6) 

(5) 

(4) 

(3) 

(2) 

(1) 




Coupled w / Data Block & Trend Indicator 






R8 

10 

12 

3 

0 

0 

0 

5.28 

25 

0.66 

R7 

7 

15 

3 

0 

0 

0 

5.16 

25 

0.61 

Coupled Only & Trend Indicator 







R5 

0 

20 

5 

0 

0 

0 

4.80 

25 

0.40 

R6 

1 

16 

6 

2 

0 

0 

4.64 

25 

0.69 

No trend indicator 








R4 

2 

11 

11 

0 

1 

0 

4.52 

25 

0.81 

R3 

2 

10 

11 

2 

0 

0 

4.48 

25 

0.75 

R1 

1 

11 

8 

5 

0 

0 

4.32 

25 

0.84 

R2 

1 

10 

11 

2 

1 

0 

4.32 

25 

0.84 
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“Please rate your level of confidence in the elements presented in the scenario if this 
were an actual APS-DPA operation.” 


Table 10-25: Overall Confidence Level 


Very 

Run Confi- 
dent 

Confi- 

dent 

Somewhat 

Confi- 

dent 

Not 

Very 

Confi- 

dent 

Not 
Confi- 
dent 
at All 

Rating 

Avg. 

N 

SD 

Wt. (5) 

(4) 

(3) 

(2) 

(1) 




Coupled w / Data Block & Trend Indicator 





R7 9 

14 

2 

0 

0 

4.28 

25 

0.60 

R8 9 

12 

4 

0 

0 

4.20 

25 

0.69 

Coupled Only & Trend Indicator 






R5 0 

20 

5 

0 

0 

3.80 

25 

0.40 

R6 0 

19 

6 

0 

0 

3.76 

25 

0.43 

No trend indicator 







R4 2 

14 

7 

2 

0 

3.64 

25 

0.74 

R2 1 

15 

7 

2 

0 

3.60 

25 

0.69 

R3 2 

13 

8 

2 

0 

3.60 

25 

0.75 

R1 0 

12 

11 

2 

0 

3.40 

25 

0.63 


“Please give an overall rating of the intuitiveness and user-friendliness of the 
combination of the display elements presented during the scenario you just completed:” 


Table 10-26: Overall Intuitiveness 


Run 

Very 

Intui- 

tive 

Intui- 

tive 

Moder- 

ately 

Intui- 

tive 

Not 

Very Intui- 
tive 

Not at 
All Intui- 
tive 

Rating 

Avg. 

N 

SD 

1 Wt. 

(5) 

(4) 

(3) 

(2) 

(1) 




Coupled w/ Data Block & Trend Indicator 
R7 10 13 1 0 

0 

4.38 

24 

0.56 

R8 

9 

12 

4 

0 

0 

4.20 

25 

0.69 

Coupled Only & Trend Indicator 
R5 2 19 4 0 

0 

3.92 

25 

0.48 

R6 

0 

16 

5 

2 

0 

3.61 

23 

0.64 

No trend indicator 
R4 2 12 

10 

1 

0 

3.60 

25 

0.69 

R3 

2 

12 

7 

3 

0 

3.54 

24 

0.82 

R2 

1 

11 

10 

3 

0 

3.40 

25 

0.75 

R1 

0 

13 

8 

4 

0 

3.36 

25 

0.74 
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10.1.3 Comments and Suggestions for Improvement 

The written responses made by subject pilots in the post-run questionnaires are discussed 
in Section 6.5. 

10.2 Results of Final Questionnaire 

This is the response data from the final questionnaires completed by each of the subjects 
at the completion of the eight (8) experimental runs R1-R8. The final questionnaire 
differed from the post-run in its focus, with questions designed to solicit overall 
impressions of design element importance, individual element preferences, and 
suggestions for improvement of the elements. A variety of responses types were collected 
using Likert-type scales, multiple choice responses and written comments. 

10.2.1 Importance of Display Elements 

The following final questionnaire responses explored the subject’s impressions of the 
importance of each category of design element (i.e., PDS target speed, PDS mode 
annunciation, etc.). The tabulated results, shown in rank order by rating average for each 
category or characteristic, illustrate the relative importance of the information which was 
intended to be provided by the design elements during the spacing simulations. The 
results of the importance of display elements responses are discussed in 6.2. 

“Using the rating scales, please indicate which of the following should be required in 
perfonning the spacing operation:” 


Table 10-27: Required Elements for Spacing Operation 


Element 

Very 

Imp. 

Imp. 

Mod. 

Imp. 

Little 

Imp. 

Not 

Imp. 

Rating 

Avg. 

N 

SD 

Weiqht 

(5) 

(4) 

(3) 

(2) 

(1) 




PDS target speed indicator 

23 

2 

0 

0 

0 

4.92 

25 

0.27 

Target aircraft depiction 

20 

3 

2 

0 

0 

4.72 

25 

0.60 

PDS mode annunciation 

15 

5 

4 

1 

0 

4.36 

25 

0.89 

Trend indicator 

13 

6 

4 

0 

2 

4.12 

25 

1.18 



“Using the rating scales, please rate how important it is to be able to determine the 
following during a spacing operation.” 


Table 10-28: PDS Target Speed Characteristic Importance 


Element 

Characteristic 

Very 

Impor- 

tant 

Impor- 

tant 

Mod. 

Impor- 

tant 

Little 

Impor- 

tance 

Not 

Impor- 

tant 

Rating 

Avg. 

N 

SD 

Weight 

(5) 

(4) 

(3) 

(2) 

(1) 




Current 
target speed 

23 

2 

0 

0 

0 

4.92 

25 

0.27 

Changes to 
target speed 

21 

3 

1 

0 

0 

4.80 

25 

0.49 


Table 10-29: Target Aircraft Display Characteristics Importance 


Element 

Characteristic 

Very 

Impor- 

tant 

Impor- 

tant 

Mod. 

Impor- 

tant 

Little 

Impor- 

tance 

Not 

Impor- 

tant 

Rating 

Avg. 

N 

SD 

Weight 

(5) 

(4) 

(3) 

(2) 

(1) 




Ownship position 
relative to 
spacing aircraft 

22 

3 

0 

0 

0 

4.88 

25 

0.32 

Active spacing 
aircraft 

21 

4 

0 

0 

0 

4.84 

25 

0.37 

How many 
(one or two) 

18 

5 

2 

0 

0 

4.64 

25 

0.62 

Spacing 

deviation(s) 

10 

10 

3 

2 

0 

4.12 

25 

0.91 

Aircraft ID(s) 

7 

10 

7 

1 

0 

3.92 

25 

0.84 

Assigned 

runway(s) 

5 

11 

7 

1 

1 

3.72 

25 

0.96 

Aircraft type 

(i.e. , 757, hvy, etc.) 

5 

10 

7 

3 

0 

3.68 

25 

0.93 

Spacing Interval type 

4 

11 

6 

4 

0 

3.60 

25 

0.94 






Table 10-30: PDS Mode Characteristic Importance 


Element 

Characteristic 

Very 

Impor- 

tant 

Impor- 

tant 

Mod. 

Impor- 

tant 

Little 

Impor- 

tance 

Not 

Impor- 

tant 

Rating 

Avg. 

N 

SD 

Weight 

( 5 ) 

( 4 ) 

( 3 ) 

(2) 

(1) 




Current mode 

13 

6 

5 

1 

0 

4.24 

25 

0.91 

Changes to mode 

11 

10 

2 

2 

0 

4.20 

25 

0.89 

Next Mode in 
Sequence 

6 

7 

5 

6 

1 

3.44 

25 

1.20 


10.2.2 Display Element Individual Preferences 

In the following final questionnaire responses, the subjects were asked to express a 
preference for design element options in each of the four categories listed (PDS target 
speed, PDS mode annunciation, target aircraft depiction and trend indication.) If subjects 
preferred not to see the design element at all, they could answer “none”. Supporting 
comments about these preferences were captured later in the final questionnaire. The 
results are tabulated in preference order shown below. At the end of this question, 
subjects were asked if their answer would change if considering the design element 
options in combination, rather than individually. 

“Please indicate if you have a preference in each of the four categories of display 
elements listed below:” 


Table 10-31: PDS Target Speed Preference 





Speed bug 

52.0% 

13 

Text 

48.0% 

12 

None 

0.0% 

0 

Total 

100.0% 

25 


Table 10-32: PDS Mode Annunciation Preference 


Element 

Option 

Response 

Percentage 

Response 

Count 

FMA 

60.0% 

15 

Text 

40.0% 

10 

None 

0.0% 

0 

Total 

100.0% 

25 
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Table 10-33: Target Aircraft Display Preference 


Element 

Option 

Response 

Percentage 

Response 

Count 

Coupled w/data block 

96.0% 

24 

Coupled only 

4.0% 

1 

None 

0.0% 

0 

Total 

100.0% 

25 


Table 10-34: Trend Indicator Preference 


Element 

Option 

Response 

Percentage 

Response 

Count 

Conformance box 

68.0% 

17 

Deviation scale 

20.0% 

5 

None 

12.0% 

3 

Total 

100.0% 

25 


10.2.3 Combined Display Element Preferences 

“Do your preferred display elements differ when considered in combination than 
individually ?” 

Eight (8) of the twenty- five subjects responded “yes” to this question. However, only one 
of the eight actually changed any of the preferred element selections. These responses 
indicate that the question may have been confusing and misinterpreted by at least seven 
of the subjects. The one subject who changed the response for the combined elements did 
so as follows: 





Table 10-35 Individual Subject Combined vs. Individual Preferences 


Display 

Element 

Individual 

Option 

Pref. 

Combined 

Element 

Pref. 

PDS Target 
Speed 

Text 

Text 

PDS Mode 
Annunciation 

FMA 

Text 

Target 

Aircraft 

Depiction 

Coupled 

w/Data 

Coupled 

w/Data 

Trend 

Indicator 

Deviation 

Scale 

Deviation 

Scale 


10.2.4 Comments and Suggestions for Improvement 

The written responses made by subject pilots in the final questionnaire are discussed 
Section 6.5. 



11 Appendix B: Supplemental Design Concepts for Fully Integrated 
Implementation 

11.1 Engine Instrument & Crew Alerting System/Multifunction Display 

The Engine Instrument & Crew Alerting System (EICAS/MFD) implements several 
functions that can be adapted to manage a datalinked APS-DPA clearance in the fully 
integrated concept. The interface displays datalinked messages from an Air Traffic 
Service Provider (ATSP). If the message is a clearance, an interface is provided to 
transfer datalinked clearance parameters to an on-board application in the Flight 
Management Computer (FMC), such as ASTAR. A facility is provided for the crew to 
accept or reject the clearance via datalink. The EICAS/MFD is also generally used to 
display infonnation from on-board applications, especially off-nominal conditions. These 
annunciations are in the form of alphanumeric message(s) which contains specific 
information about the condition and its associated priority, often associated with an aural 
indication. 

The ASTOR used for the previous HITL experiment [5] was modified to support the fully 
integrated APS-DPA spacing scenarios used for this study. Simulations were created to 
generate the datalinked clearances for the spacing scenario (see Table 5-1: Spacing 
Scenario Clearance Parameters). The ASTOR was modified to parse the uplinked 
clearance and provide the spacing parameters to ASTAR. The spacing parameters could 
then be viewed and verified via the PDS main menu page in the MCDU (see Section 
1 1.3). Due to time constraints, the off-nominal condition outputs from ASTAR were not 
implemented, and no off-nominal scenarios were generated for the usability study. 

Examples of the ASTOR implementation of the EICAS/MFD for APS-DPA clearance 
management are shown in the following illustrations. Figure 11-1 shows a spacing 
clearance received via CPDLC (Controller Pilot Data Link Clearance) displayed on the 
EICAS/MFD. Note that the length of the clearance (greater than 128 characters) 
necessitated that it be viewed on the MFD, also indicated by the “LARGE ATC 
UPLINK” message on the EICAS. 




11.2 Mode Control Panel 

In the proposed concept, the Mode Control Panel (MCP) allowed the crew to control the 
engagement and disengagement of PDS speed guidance to the autopilot. The MCP 
provides control of the autopilot, flight director, altitude alert, and auto-throttle systems, 
and is used to select and activate Autopilot Flight Director System (AFDS) modes, 
establish altitudes, speeds and climb/descent profdes. There were two concepts 
considered for the MCP and its associated control logic: 1) addition of new “PDS” mode 
control select to the MCP, and; 2) modification of MCP control logic to make PDS mode 
a sub-mode of the existing VNAV mode. 

In considering the limited time for training subject pilots prior to the experimental 
session, it was decided that the first concept would be potentially less confusing to 
subject pilots who were already familiar with the operation of the existing VNAV mode 
of the AFDS. As the focus of the experiment was primarily on design elements for the 
Primary Flight and Navigation displays, only the first concept was implemented in the 
modified ASTOR. 

The implementation of the first concept required the straightforward addition of a new 
“PDS” mode to the MCP to implement the engagement of PDS speed guidance for 
spacing operations. Its implementational advantage was that the logic that controls its 
operation was constructed to include little or no interaction with existing AFDS modes. 
The diagram in Figure 11-2 shows the proposed logic to control the engagement of PDS 
guidance via the MCP, and describes the progression of nominal PDS modes for a typical 
spacing operation. The diagram does not include the logic for degraded mode 
annunciations due to PDS off-nominal conditions. 
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Figure 11-3 shows an example of how the modified MCP appeared with the new PDS 
mode select (highlighted by arrow) when PDS guidance was engaged (indicated by the 
green light.) 



Figure 11-3: MCP with PDS Mode Select Added 


The implementation of PDS mode as a sub-mode of the VNAV mode is conceptually 
consistent with the operation of APS-DPA, which provides profile-driven speed 
corrections to a CDA. It is also consistent with the paradigm of dividing VNAV 
operation into other sub-modes (i.e., VNAV SPD, VNAV PTH, VNAV ALT). This 
concept would require the modification of the logic that controls the VNAV mode and 
(possibly) its interaction with other existing AFDS modes. In any practical 
implementation of APS-DPA in the cockpit, some modifications to the AFDS are bound 



to occur anyway, and it is believed that this type of software-only modification may have 
an economic advantage over the implemented concept described above. 

11.3 Multifunction Control and Display Unit 

In the fully integrated concept, the Multifunction Control and Display Unit (MCDU) 
provided an interface to the on-board ASTAR application for selecting modes of 
operation, entering spacing parameters, validating spacing clearances, monitoring the 
status of an in-progress APS-DPA operation, and terminating ASTAR if necessary. The 
ASTOR implementation for this study developed several new MCDU menu pages to 
facilitate the required additional functions. 

The ASTOR implementation added three (3) new PDS-specific pages to the MCDU. A 
calling page was modified to allow the main PDS menu to be invoked (see Figure 1 1-4). 
From the main PDS menu, two sub-menu pages for display and control of approach 
parameters (“PDS APPR DATA”) and PDS profile mode (“PDS Profile”) were defined. 
PDS profile mode is discussed in the APS ConOps, but was not implemented or 
evaluated in this study. 



Within the new pages, PDS-specific status and control functions were assigned to the 
twelve (12) “soft” keys aligned vertically on either side of the display bezel. The soft 
keys are arranged in two vertical rows of six (6) each on the left and right, referred to as 
1L through 6L for the left side, and 1R through 6R on the right side. Per MCDU 






convention, a left or right pointer to any of the twelve bezel keys indicated that a 
programmable action was performed by pressing that key. The action was either 
indicated by the associated label (i.e., “TERMINATE”) or was implied as an editing 
action in the case of an alphanumeric parameter. The absence of the pointer indicated the 
parameter was non-editable. Refer to Table 11-1 and Table 11-2 for a description of the 
soft key functions defined for the PDS main menu menu and approach data pages. 

The main PDS menu page provided most of the necessary control and monitoring 
functions for the APS-DPA operation; the “PDS APPR DATA” page provided 
supplemental information and a means for the crew to enter, modify or monitor ASTAR 
parameters including landing runway assignment(s) for target aircraft, target aircraft 
approach speed(s), minimum in-trail distance behind target aircraft, and ownship 
approach speed. 


Table 11-1: PDS Main Menu Page 


MCDU 
Soft Key 

Label 

Functional Description 

1 L 

< PRECISION 
REFAC 

Precision REFerence AirCraft (target aircraft); auto-loaded or pilot-entered 

ID; when magenta, 

indicates active spacing aircraft. 

2L 

< SPACING 
INTERVAL 

Auto-loaded or pilot-entered interval in seconds or nm; 
when magenta, indicates PDS speed guidance engaged 
using this parameter. 

3L 

SPACING DEV 

ASTAR-provided deviation in seconds. 

4L 

< STA 

Auto-loaded or pilot-entered STA in Zulu format; when magenta, indicates 
PDS speed guidance engaged using this parameter; when REFAC ADS-B 
data is acquired, STA will no longer be displayed. 

5L 

< PROFILE* 

In some implementations, invokes PDS Profile menu page. 

6L 

< APPR DATA 

Invoke PDS Approach Data menu page. 

1 R 

NOT REFAC > 

No Closer Than target aircraft; auto-loaded or pilot-entered ID; when 
magenta, indicates active spacing aircraft. 

2R 

SPACING 
INTERVAL > 

Auto-loaded or pilot-entered interval in seconds or nm; when magenta, 
indicates PDS speed guidance engaged using this parameter. 

3R 

SPACING DEV 

ASTAR-provided deviation in seconds or nm. 

4R 

STA DEV 

ASTAR-provided deviation in seconds; when target aircraft ADS-B data is 
acquired, STA DEV will no longer be displayed. 

5R 

ERASE > 

Only appears if parameters have been changed; erase all page 
modifications (revert to previous unmodified parameters). 

6R 

TERMINATE > 

Only appears if ASTAR is active; terminate ASTAR until new parameters 
entered; disengage PDS from AFDS (see MCP description in 1 1 .2). 


*PDS Profile mode was not implemented for this study, but is discussed in Appendix D: Additional 
Interface Concepts for APS-DPA 





Table 1 1-2: PDS Approach Data Menu 


MCDU Soft 
Key 

Label 

Function 

1 L 

PRECISION 

Precision REFerence AirCraft (target aircraft); AC ID from PDS main 

REFAC 

page. 

2L 

< APPROACH 

Auto-loaded from ASTAR from ADS-B on-condition report or pilot- 

SPEED 

entered. 

3L 

< MIN DISTANCE 

*Auto-loaded from ASTAR or pilot-entered 

4L 

< LANDING RWY 

Landing runway for precision REFAC from one of following: 
auto-loaded from clearance; pilot-entered, or; auto-loaded from ABS- 
B on-condition report. 

5L 

Unused 


6L 

< PDS 

Invoke PDS main menu page. 

1 R 

NCT REFAC 

No Closer Than target aircraft; AC ID from PDS main page. 

2R 

APPROACH 

Auto-loaded from ASTAR from ADS-B on-condition report or pilot- 

SPEED > 

entered. 

3R 

MIN DISTANCE > 

*Auto-loaded from ASTAR or pilot-entered. 

4R 

LANDING RWY > 

Landing runway for precision target aircraft from one of following: 
auto-loaded from clearance; pilot-entered, or; auto-loaded from ABS- 
B on-condition report. 

5R 

OWNSHIP APPR 
SPEED > 

Auto-loaded from profile or pilot-entered. 

6R 

ERASE > 

Only appears if parameters have been changed; erase all page 
modifications (revert to previous unmodified parameters). 


*Minimum Distance represents the minimum wake turbulence avoidance in-trail distance for the category 
of aircraft involved. In an ADS-B environment, this parameter could be automatically entered via an on- 
board “look-up” table. 


The following are pictorial examples of MCDU PDS menu pages that were implemented 
for the study, shown in chronological order as a typical spacing scenario progressed. 

Figure 11-5 shows the PDS Main menu as it appeared after loading the received CPDLC 
spacing clearance to the FMC via the EICAS/MFD interface. Unpopulated fields for 
Spacing Deviation and STA Deviation indicate that ASTAR had not yet provided spacing 
guidance from the input spacing parameters, but was in the process of computing them. 
The “PDS-MOD” heading indicate parameters were changed but not executed, also 
indicated by the lit “EXECute” button on the MCDU. 





PDS - MOD 




Figure 11-5: PDS Main Menu Page; Clearance Loaded 


Figure 1 1-6 shows the PDS Approach Data page as it appeared after loading the received 
CPDLC spacing clearance to the FMC. The approach speed, minimum distance and 
landing runway for the precision target (NASA 893) have been provided by ASTAR, 
indicating ADS-B on-condition reports have been received from NASA 893. The same 
parameters for the no closer than (NCT) target (NASA 859) are not populated, but would 
be automatically filled by ASTAR when ADS-B on-condition reports from the target 
aircraft were received. The ownship approach speed has been loaded from the current 
profile being flown by the FMC. 
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Figure 1 1-6: PDS Approach Data Page 






Figure 11-7 shows the PDS main menu page after ASTAR has determined that the 
precision target (NASA 893) is the active spacing aircraft (responsible for the current 
PDS target speed), indicated by the “PRECISION REFAC” and “SPACING INTVL” in 
magenta. ASTAR also maintained the STA Deviation parameter throughout the 
operation, though it did not provide spacing guidance after ADS-B reports were received 
for the target aircraft. 



Figure 11-7: PDS Main Menu; ASTAR Guidance Active 


Figure 11-8 is similar to Figure 1 1-7, except that the test operator had entered the spacing 
parameters for the NCT target aircraft in the spacing scenario (NASA 859, NCT 90 sec), 
and had executed the entire combined spacing clearance. Note that the “EXECute” button 
light is extinguished, and the “PDS-MOD” heading has reverted to “PDS”. Spacing 
deviations from ASTAR now appear for the precision and NCT target aircraft and the 
STA, and the precision target is still the active target, indicated in magenta. 







Figure 11-8: PDS Main Menu; NCT Interval Added 


Figure 1 1-9 is similar to Figure 11-8 except that ASTAR determined that the NCT target 
(NASA 859) was the active spacing aircraft, indicated in magenta. 



Figure 1 1-9: PDS Main Menu; NCT REFAC Active 
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12 Appendix C: Electronic Flight Bag Interface Concepts for APS-DPA 

12.1 Description of Design Concept 

The goal for the EFB version of the APS-DPA interface was to propose concepts 
allowing pilots to perform the following required functions: 

1 . initialize ASTAR with spacing clearance parameters; 

2. verify validity of spacing clearance parameters before accepting; 

3. display PDS target speed necessary to achieve spacing interval(s); and 

4. monitor progress of APS-DPA, including; 

a. PDS operation mode; 

b. target aircraft position and infonnation; 

c. spacing deviation and trend infonnation; and 

d. off-nominal indications. 

It was proposed to develop an EFB-based application to provide the required functions 
(the application is referred to as Pair Dependent Speed or “PDS” for this discussion.) 
Pilots would initialize PDS with spacing parameters received via voice clearance. Once 
verified and executed, PDS would provide the target speed necessary to achieve the 
spacing interval, which the pilot would use to manually set the speed of the aircraft. PDS 
would provide strategic infonnation about APS-DPA by implementing displays similar to 
those developed for those on the ND in the fully integrated concept. 

In order to function, the embedded ASTAR algorithm in PDS would require access to 
data available from the aircraft (ARINC429) data bus, such as the profile being flown by 
the aircraft and the ADS-B on-condition reports from the target aircraft. This would 
necessitate a minimum class 2 EFB for read-only access to this data; it was not 
envisioned that PDS would require write access to ARINC429 for the implementation of 
this concept. 

Since EFBs differ in user control interfaces, a representative implementation is used for 
illustration purposes that employs a combination of soft keys along the bezels, pre- 
defined keys above and below the screen and a touch screen interface. It is assumed that 
the concepts shown here could be easily transferred to other EFB implementations 

12.1.1 PDS Home Page 

Figure 12-1 shows a concept of the PDS application home page on the EFB. This page 
would be accessible by navigating through the EFB main menu to the PDS application. In 
this concept, the PDS home page is used to monitor the APS-DPA operation. 




Figure 12-1: EFB PDS Home Page Concept 


The top of the PDS home page displays PDS mode and target speed. The crew would be 
alerted to target speed changes by highlighting the “Speed Cmd” area (with a box or 
shading) for ten seconds. Similarly, changes to “Mode” would be highlighted for ten 
seconds. 

Below this, an area of the EFB screen is dedicated to displaying clearance parameters and 
ASTAR-supplied information such as active target aircraft (shown in magenta), spacing 
deviation (in seconds) from the assigned interval for each target aircraft, and spacing 
deviation from the original STA. Also, a soft key is implemented to access a sub-menu to 




enter the spacing clearance parameters. Another soft key is implemented to tenninate 
PDS speed guidance and clear the current spacing parameters. 

The area below the clearance spacing data values implements a moving map. The concept 
for this map draws directly from the concepts for the Navigation display (ND) in the fully 
integrated implementation. The ownship aircraft symbol is a white triangle at the bottom 
center. The current magnetic heading is displayed at the top of the map for reference. A 
soft key is implemented to control the range of the map (set to 40 NM in the illustration). 
The target aircraft are shown as green double chevrons. An example of the trend indicator 
is also shown. If the stagger interval display concept were implemented, it would also be 
shown here. 

12.1.2 Spacing Clearance Sub-Page 

Figure 12-2 is a concept drawing of the PDS Spacing Clearance sub-menu. This menu 
facilitates entry or amendment of the spacing clearance parameters. A touch screen 
keyboard interface is shown which would simplify entry of alphanumeric clearance 
parameters. 




Figure 12-2: EFB Spacing Sub-Page Concept 









13 Appendix D: Additional Interface Concepts for APS-DPA 


Additional interface concepts were considered for the situations which might occur 
during APS-DPA operations. These concepts were not evaluated in this study, because 
schedule and resource limitations did not permit the development of alternative concepts 
and/or implementation in the ASTOR. The concepts are captured in this appendix to be 
considered in the future. 

13.1 Stagger Interval Depiction 

It was considered possible for a step change in the PDS target speed to occur if ASTAR 
transitioned from using the assigned spacing interval to using the stagger interval for a 
target aircraft on approach to a parallel runway. The APS ConOps [7] describes this 
situation in more detail. While not an abnormal situation, abrupt changes in PDS target 
speed might cause the crew to question the new speed unless the context for the change 
was also displayed. For this reason, a concept was developed to inform the crew that the 
stagger interval either was or would soon be the governing spacing interval for ASTAR. 

ASTAR’s transition to stagger interval was considered strategic information, and a 
concept was developed to display this situation on the Navigation display (ND). The 
concept consists of a white dashed line, which pictorially represents the stagger geometry 
relative to the target aircraft on the parallel runway approach. This depiction is shown in 
Figure 13-1. The concept was intended to show the crew where the intercept of the 
stagger interval would occur, and in doing so, provide the necessary context for the PDS 
target speed change. 
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Figure 13-1: Stagger Interval Depiction 



Figure 13-2 shows the main PDS menu page as would appear after ASTAR had 
transitioned to using the stagger interval. In this depiction, the precision target is the 
active spacing aircraft, and ASTAR has changed the spacing interval from the assigned 
interval (in seconds) to the stagger interval. The stagger interval would have been pre- 
programmed for this particular dependent parallel approach. 
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Figure 13-2: ASTAR Transition to Stagger Interval 


13.2 PDS Speed Reversion Mode 

PDS Speed Reversion mode is an off-nominal condition indicating PDS speed guidance 
is no longer valid due to invalid target aircraft data. This condition could occur because 
ADS-B on-condition reports from the target aircraft are no longer being received, and the 
condition could be temporary or permanent. The annunciation of PDS Speed Reversion 
mode would inform pilots that PDS’s ability to provide target speed guidance was on 
hold. The APS ConOps describes this condition in more detail. 

A concept was developed to display PDS Speed Reversion mode for the PDS mode 
annunciation option in which the FMA was used to display PDS modes. In this concept, a 
yellow line would be drawn through the PDS Paired mode annunciation in the FMA for 
as long as the condition persists, or until ASTAR was terminated by the crew. (PDS 
Paired mode is the only PDS mode that requires ADS-B reports from the target aircraft.) 
See Table 13-1 for an illustration of this concept. 








Table 13-1: PDS Speed Reversion Mode Concept 
PDS FMA Example Description FMA Depiction 

PDS Mode Degraded PDS off-nominal condition (i.e., Speed 

Reversion mode); EICAS message provides 
additional detail; pilot action may be required to 
disengage PDS 

13.3 PDS Profile Mode 

PDS Profile mode provides a capability that leverages ASTAR’s ability to provide target 
speeds while flying any programmed profile. While these target speeds would be 
identical to the speeds contained in the profile in most cases, ASTAR is programmed to 
use stabilized approach speeds on final approach as the target speed, and may therefore 
be a preferable type of speed guidance for this phase of flight. The APS ConOps 
describes Profile mode in more detail. 

A concept was developed to allow the pilot to control the use of Profile mode via the 
MCDU. Figure 13-3 shows a typical PDS main menu page on the MCDU with the profile 
mode select implemented for soft key 5L. Figure 13-4 shows the The PDS-PROFILE 
sub-page that would be used by the crew to control, execute and monitor the PDS Profile 
mode of ASTAR operation. The only choices available are to “Enable” the PDS Profile 
mode, or to return to the main PDS menu. Figure 13-5 shows the PDS-PROFILE menu 
after PDS Profile mode has been enabled. In this example, the profile name appears in 
magenta, indicating Profile mode has been engaged by selecting PDS mode on the MCP. 
Refer to Table 13-2 for a description of this sub-page’s status and control functions. 




Figure 13-3: PDS Main Menu with Profile Mode Selection 
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Table 13-2: PDS Profile Menu Page 


MCDU Soft Key 

Label 

Function 

1 L 

Unused 


2L 

Unused 


3L 

Unused 


4L 

Unused 


5L 

Unused 


6L 

< PDS 

Invoke PDS main menu page. 

1 R 

PROFILE SPEEDS: 

Used to enable PDS Profile mode; when enabled and 

ENABLE > 

engaged, 

<Profile Name> of profile in use appears in magenta. 

2R 

Unused 


3R 

Unused 


4R 

Unused 


5R 

Unused 


6R 

TERMINATE > 

Terminate ASTAR until new parameters entered; 
disengage PDS from AFDS (see MCP description in 
11.2). 





14 Appendix E: ASTOR Implementation Issues 

This section documents ASTOR implementation issues or problems noted during the 
conduct of the experiment. These anomalies were not corrected in the interest of making 
identical presentions to the subject pilots. Any anomalies which may affected the 
outcome of the experiment have been noted in the main body of the report. The 
anomalies are documented here to be of use in any subsequent efforts to leverage the 
ASTOR implementation used for this experiment in future studies or simulations. 

14.1 PDS Mode Annunciation 

For the text mode option of this design element, it was noted by one subject pilot that 
when PDS mode appeared by itself in text (i.e., without PDS target speed text), that the 
display simply indicated the mode without also indicating “PDS”. For instance, in PDS 
Paired mode, the text on the PFD indicated “PAIR” instead of “PDS PAIR”. This is a 
result of the ASTOR implementation, which removed “PDS” from the text display when 
PDS target speed was configured to display the speed bug option. 

For the FMA option of this design element, several subject pilots noted that the specific 
VNAV mode currently in use (i.e., VNAV PTH, VNAV SPD, VNAV ALT) was never 
shown. This is a result of the ASTOR implementation ignoring the VNAV mode, and 
simply displaying ‘VNAV’ in the pitch mode segment. This VNAV mode is obviously 
important infonnation and should also be shown in the pitch segmant. 

For the FMA mode of this design element, one subject pilot noted that in the FMA 
segment shared by VNAV and PDS modes, a change to VNAV or PDS mode would 
cause a box to be drawn around the entire segment. This obscured the meaning about 
whether the VNAV or PDS mode had caused the change. This is a result of the ASTOR 
implementation not creating a separate box for each portion of the shared pitch segment. 

14.2 Target Aircraft Depiction 

For the “coupled with data block” option of this design element, several subject pilots 
noted that the active target aircraft data block location was in conflict with the existing 
implementation in the Boeing glass cockpit for the display of NAVAID data, normally 
selected using the EFIS control panel. This is a result of the ASTOR implementation 
ignoring this potential conflict. 

Also, the data block implementation always contained the text “PRI”, which was meant 
to indicate that the the associated infonnation in the data block was for the active spacing 
aircraft. This infonnation was unnecessary, since the data block only appeared in 
association with the active spacing aircraft. This is a result of the ASTOR implementation 
using an older concept which would have been able to show a data block for either target 
aircraft. (The “PRI” indicated “Primary”, which was a tenn fonnerly used for the active 
spacing aircraft). 



14.3 Trend Indicators 


For the conformance box option of this design element, it was noted that the range- 
sensitive braces used to depict the box could potentially be driven off-scale by the ND 
range select. The convention for depicting off-scale CDTI symbols is to depict a half- 
symbol at the edge of the range arc at the appropriate bearing. Adapting this convention 
to the conformance box braces would have prevented the depiction from disappearing at 
lower ND range settings. 
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